pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Flush Memoize cache when non-key parameters change
8+ messages / 3 participants
[nested] [flat]

* pgsql: Flush Memoize cache when non-key parameters change
@ 2021-11-24 01:56 David Rowley <drowley@postgresql.org>
  0 siblings, 0 replies; 8+ messages in thread

From: David Rowley @ 2021-11-24 01:56 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Flush Memoize cache when non-key parameters change

It's possible that a subplan below a Memoize node contains a parameter
from above the Memoize node.  If this parameter changes then cache entries
may become out-dated due to the new parameter value.

Previously Memoize was mistakenly not aware of this.  We fix this here by
flushing the cache whenever a parameter that's not part of the cache
key changes.

Bug: #17213
Reported by: Elvis Pranskevichus
Author: David Rowley
Discussion: https://postgr.es/m/17213-988ed34b225a2862@postgresql.org
Backpatch-through: 14, where Memoize was added

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/1050048a315790a505465bfcceb26eaf8dbc7e2e

Modified Files
--------------
src/backend/executor/nodeMemoize.c      | 38 ++++++++++++++++++++++++++++++++
src/backend/nodes/bitmapset.c           |  2 ++
src/backend/optimizer/plan/createplan.c | 10 ++++++---
src/backend/optimizer/util/clauses.c    | 31 ++++++++++++++++++++++++++
src/include/nodes/execnodes.h           |  2 ++
src/include/nodes/plannodes.h           |  1 +
src/include/optimizer/clauses.h         |  2 ++
src/test/regress/expected/memoize.out   | 39 +++++++++++++++++++++++++++++++++
src/test/regress/sql/memoize.sql        | 20 +++++++++++++++++
9 files changed, 142 insertions(+), 3 deletions(-)



^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* pgsql: Flush Memoize cache when non-key parameters change
@ 2021-11-24 01:57 David Rowley <drowley@postgresql.org>
  2021-11-24 02:18 ` Re: pgsql: Flush Memoize cache when non-key parameters change David Rowley <dgrowleyml@gmail.com>
  0 siblings, 1 reply; 8+ messages in thread

From: David Rowley @ 2021-11-24 01:57 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Flush Memoize cache when non-key parameters change

It's possible that a subplan below a Memoize node contains a parameter
from above the Memoize node.  If this parameter changes then cache entries
may become out-dated due to the new parameter value.

Previously Memoize was mistakenly not aware of this.  We fix this here by
flushing the cache whenever a parameter that's not part of the cache
key changes.

Bug: #17213
Reported by: Elvis Pranskevichus
Author: David Rowley
Discussion: https://postgr.es/m/17213-988ed34b225a2862@postgresql.org
Backpatch-through: 14, where Memoize was added

Branch
------
REL_14_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/f94edb06ab60545a0e9e90aedc39e3b81b2b9e3f

Modified Files
--------------
src/backend/executor/nodeMemoize.c      | 38 ++++++++++++++++++++++++++++++++
src/backend/nodes/bitmapset.c           |  2 ++
src/backend/optimizer/plan/createplan.c | 10 ++++++---
src/backend/optimizer/util/clauses.c    | 31 ++++++++++++++++++++++++++
src/include/nodes/execnodes.h           |  2 ++
src/include/nodes/plannodes.h           |  1 +
src/include/optimizer/clauses.h         |  2 ++
src/test/regress/expected/memoize.out   | 39 +++++++++++++++++++++++++++++++++
src/test/regress/sql/memoize.sql        | 20 +++++++++++++++++
9 files changed, 142 insertions(+), 3 deletions(-)



^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: pgsql: Flush Memoize cache when non-key parameters change
  2021-11-24 01:57 pgsql: Flush Memoize cache when non-key parameters change David Rowley <drowley@postgresql.org>
@ 2021-11-24 02:18 ` David Rowley <dgrowleyml@gmail.com>
  2021-11-24 04:31   ` Re: pgsql: Flush Memoize cache when non-key parameters change Michael Paquier <michael@paquier.xyz>
  0 siblings, 1 reply; 8+ messages in thread

From: David Rowley @ 2021-11-24 02:18 UTC (permalink / raw)
  To: David Rowley <drowley@postgresql.org>; +Cc: pgsql-committers@lists.postgresql.org

On Wed, 24 Nov 2021 at 14:57, David Rowley <drowley@postgresql.org> wrote:
> Flush Memoize cache when non-key parameters change

I'm currently looking into why the buildfarm does not like this.

David





^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: pgsql: Flush Memoize cache when non-key parameters change
  2021-11-24 01:57 pgsql: Flush Memoize cache when non-key parameters change David Rowley <drowley@postgresql.org>
  2021-11-24 02:18 ` Re: pgsql: Flush Memoize cache when non-key parameters change David Rowley <dgrowleyml@gmail.com>
@ 2021-11-24 04:31   ` Michael Paquier <michael@paquier.xyz>
  2021-11-24 07:27     ` Re: pgsql: Flush Memoize cache when non-key parameters change David Rowley <dgrowleyml@gmail.com>
  0 siblings, 1 reply; 8+ messages in thread

From: Michael Paquier @ 2021-11-24 04:31 UTC (permalink / raw)
  To: David Rowley <dgrowleyml@gmail.com>; +Cc: David Rowley <drowley@postgresql.org>; pgsql-committers@lists.postgresql.org

On Wed, Nov 24, 2021 at 03:18:53PM +1300, David Rowley wrote:
> I'm currently looking into why the buildfarm does not like this.

From what I can see, there are two separate problems here:
- force_parallel_mode = regress.
- -DRELCACHE_FORCE_RELEASE and/or -DCATCACHE_FORCE_RELEASE.

Regards,
--
Michael

Attachments:

  signature.asc (application/pgp-signature, 832B) download

^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: pgsql: Flush Memoize cache when non-key parameters change
  2021-11-24 01:57 pgsql: Flush Memoize cache when non-key parameters change David Rowley <drowley@postgresql.org>
  2021-11-24 02:18 ` Re: pgsql: Flush Memoize cache when non-key parameters change David Rowley <dgrowleyml@gmail.com>
  2021-11-24 04:31   ` Re: pgsql: Flush Memoize cache when non-key parameters change Michael Paquier <michael@paquier.xyz>
@ 2021-11-24 07:27     ` David Rowley <dgrowleyml@gmail.com>
  2021-11-24 09:54       ` Re: pgsql: Flush Memoize cache when non-key parameters change David Rowley <dgrowleyml@gmail.com>
  0 siblings, 1 reply; 8+ messages in thread

From: David Rowley @ 2021-11-24 07:27 UTC (permalink / raw)
  To: Michael Paquier <michael@paquier.xyz>; +Cc: David Rowley <drowley@postgresql.org>; pgsql-committers@lists.postgresql.org

On Wed, 24 Nov 2021 at 17:31, Michael Paquier <michael@paquier.xyz> wrote:
> From what I can see, there are two separate problems here:
> - force_parallel_mode = regress.
> - -DRELCACHE_FORCE_RELEASE and/or -DCATCACHE_FORCE_RELEASE.

Thanks. I missed adding keyparamids to the copy/read/write functions.

David





^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: pgsql: Flush Memoize cache when non-key parameters change
  2021-11-24 01:57 pgsql: Flush Memoize cache when non-key parameters change David Rowley <drowley@postgresql.org>
  2021-11-24 02:18 ` Re: pgsql: Flush Memoize cache when non-key parameters change David Rowley <dgrowleyml@gmail.com>
  2021-11-24 04:31   ` Re: pgsql: Flush Memoize cache when non-key parameters change Michael Paquier <michael@paquier.xyz>
  2021-11-24 07:27     ` Re: pgsql: Flush Memoize cache when non-key parameters change David Rowley <dgrowleyml@gmail.com>
@ 2021-11-24 09:54       ` David Rowley <dgrowleyml@gmail.com>
  0 siblings, 0 replies; 8+ messages in thread

From: David Rowley @ 2021-11-24 09:54 UTC (permalink / raw)
  To: Michael Paquier <michael@paquier.xyz>; +Cc: David Rowley <drowley@postgresql.org>; pgsql-committers@lists.postgresql.org

On Wed, 24 Nov 2021 at 20:27, David Rowley <dgrowleyml@gmail.com> wrote:
>
> On Wed, 24 Nov 2021 at 17:31, Michael Paquier <michael@paquier.xyz> wrote:
> > From what I can see, there are two separate problems here:
> > - force_parallel_mode = regress.
> > - -DRELCACHE_FORCE_RELEASE and/or -DCATCACHE_FORCE_RELEASE.
>
> Thanks. I missed adding keyparamids to the copy/read/write functions.

Looks like the cause of the animals that had the plan change [1][2] is
a timing issue where tenk2 is not auto-analyzed before the Memoize
test.  Due to the lack of stats, Memoize is not considered in the
planner.

I see tenk1 is manually ANALYZEd, so I'll just change the tests to use
tenk1 instead of tenk2.

David

[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=serinus&dt=2021-11-24%2002%3A04%3A18
[2] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=flaviventris&dt=2021-11-24%2002%3A24%3A0...





^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* pgsql: Flush Memoize cache when non-key parameters change, take 2
@ 2021-11-24 10:29 David Rowley <drowley@postgresql.org>
  0 siblings, 0 replies; 8+ messages in thread

From: David Rowley @ 2021-11-24 10:29 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Flush Memoize cache when non-key parameters change, take 2

It's possible that a subplan below a Memoize node contains a parameter
from above the Memoize node.  If this parameter changes then cache entries
may become out-dated due to the new parameter value.

Previously Memoize was mistakenly not aware of this.  We fix this here by
flushing the cache whenever a parameter that's not part of the cache
key changes.

Bug: #17213
Reported by: Elvis Pranskevichus
Author: David Rowley
Discussion: https://postgr.es/m/17213-988ed34b225a2862@postgresql.org
Backpatch-through: 14, where Memoize was added

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/411137a429210e432f923264a8e313a9872910ca

Modified Files
--------------
src/backend/executor/nodeMemoize.c      | 38 ++++++++++++++++++++++++++++++++
src/backend/nodes/bitmapset.c           |  2 ++
src/backend/nodes/copyfuncs.c           |  1 +
src/backend/nodes/outfuncs.c            |  1 +
src/backend/nodes/readfuncs.c           |  1 +
src/backend/optimizer/plan/createplan.c | 10 ++++++---
src/backend/optimizer/util/clauses.c    | 31 ++++++++++++++++++++++++++
src/include/nodes/execnodes.h           |  2 ++
src/include/nodes/plannodes.h           |  1 +
src/include/optimizer/clauses.h         |  2 ++
src/test/regress/expected/memoize.out   | 39 +++++++++++++++++++++++++++++++++
src/test/regress/sql/memoize.sql        | 20 +++++++++++++++++
12 files changed, 145 insertions(+), 3 deletions(-)



^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* pgsql: Flush Memoize cache when non-key parameters change, take 2
@ 2021-11-24 10:30 David Rowley <drowley@postgresql.org>
  0 siblings, 0 replies; 8+ messages in thread

From: David Rowley @ 2021-11-24 10:30 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Flush Memoize cache when non-key parameters change, take 2

It's possible that a subplan below a Memoize node contains a parameter
from above the Memoize node.  If this parameter changes then cache entries
may become out-dated due to the new parameter value.

Previously Memoize was mistakenly not aware of this.  We fix this here by
flushing the cache whenever a parameter that's not part of the cache
key changes.

Bug: #17213
Reported by: Elvis Pranskevichus
Author: David Rowley
Discussion: https://postgr.es/m/17213-988ed34b225a2862@postgresql.org
Backpatch-through: 14, where Memoize was added

Branch
------
REL_14_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/c2dc7b9e15039ce90c0a631ff291feb81d2c8051

Modified Files
--------------
src/backend/executor/nodeMemoize.c      | 38 ++++++++++++++++++++++++++++++++
src/backend/nodes/bitmapset.c           |  2 ++
src/backend/nodes/copyfuncs.c           |  1 +
src/backend/nodes/outfuncs.c            |  1 +
src/backend/nodes/readfuncs.c           |  1 +
src/backend/optimizer/plan/createplan.c | 10 ++++++---
src/backend/optimizer/util/clauses.c    | 31 ++++++++++++++++++++++++++
src/include/nodes/execnodes.h           |  2 ++
src/include/nodes/plannodes.h           |  1 +
src/include/optimizer/clauses.h         |  2 ++
src/test/regress/expected/memoize.out   | 39 +++++++++++++++++++++++++++++++++
src/test/regress/sql/memoize.sql        | 20 +++++++++++++++++
12 files changed, 145 insertions(+), 3 deletions(-)



^ permalink  raw  reply  [nested|flat] 8+ messages in thread


end of thread, other threads:[~2021-11-24 10:30 UTC | newest]

Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-11-24 01:56 pgsql: Flush Memoize cache when non-key parameters change David Rowley <drowley@postgresql.org>
2021-11-24 01:57 pgsql: Flush Memoize cache when non-key parameters change David Rowley <drowley@postgresql.org>
2021-11-24 02:18 ` David Rowley <dgrowleyml@gmail.com>
2021-11-24 04:31   ` Michael Paquier <michael@paquier.xyz>
2021-11-24 07:27     ` David Rowley <dgrowleyml@gmail.com>
2021-11-24 09:54       ` David Rowley <dgrowleyml@gmail.com>
2021-11-24 10:29 pgsql: Flush Memoize cache when non-key parameters change, take 2 David Rowley <drowley@postgresql.org>
2021-11-24 10:30 pgsql: Flush Memoize cache when non-key parameters change, take 2 David Rowley <drowley@postgresql.org>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox