agora inbox for pgsql-admin@postgresql.org
help / color / mirror / Atom feedFrom: Scott Ribe <scott_ribe@elevated-dev.com>
To: Paul Smith* <paul@pscs.co.uk>
To: David G. Johnston <david.g.johnston@gmail.com>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: small temp files
Date: Mon, 22 Jul 2024 07:56:06 -0600
Message-ID: <6C269096-47A8-4F23-87AC-512656D4A084@elevated-dev.com> (raw)
In-Reply-To: <fe4b82fa-9907-43f9-b6ff-e1cbc768158d@pscs.co.uk>
References: <7A0C9A69-5632-4CFC-B156-53FAAAB33E26@elevated-dev.com>
<a9dee6e8-6197-d9bb-028e-6d020167bf57@jakobs.com>
<8E03B792-C389-456D-913D-4F2B0EBB903E@elevated-dev.com>
<fe4b82fa-9907-43f9-b6ff-e1cbc768158d@pscs.co.uk>
> ...with each operation generally being allowed to use as much memory as this value specifies before it starts to write data into temporary files.
So, doesn't explain the 7452-byte files. Unless an operation can use a temporary file as an addendum to work_mem, instead of spilling the RAM contents to disk as is my understanding.
> So, if it's doing lots of joins, there may be lots of bits of temporary data which together add up to more than work_mem.
If it's doing lots of joins, each will get work_mem--there is no "adding up" among operations using work_mem.
> You expect the smallest temporary file to be 128MB? I.e., if the memory used exceeds work_mem all of it gets put into the temp file at that point? Versus only the amount of data that exceeds work_mem getting pushed out to the temporary file. The overflow only design seems much more reasonable - why write to disk that which fits, and already exists, in memory.
Well, I don't know of an algorithm which can effectively sort 128MB + 7KB of data using 128MB of RAM and a 7KB file. Same for many of the other operations which use work_mem, so yes, I expected spill over to start with 128MB file and grow it as needed. If I'm wrong and there are operations which can effectively use temp files as adjunct, then that would be the answer to my question. Does anybody know for sure that this is the case?
view thread (8+ messages) latest in thread
Message-ID: <6C269096-47A8-4F23-87AC-512656D4A084@elevated-dev.com>
Permalink: ../6C269096-47A8-4F23-87AC-512656D4A084@elevated-dev.com/
Also on: postgresql.org/message-id/6C269096-47A8-4F23-87AC-512656D4A084@elevated-dev.com
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: scott_ribe@elevated-dev.com, paul@pscs.co.uk, david.g.johnston@gmail.com, pgsql-admin@lists.postgresql.org
Subject: Re: small temp files
In-Reply-To: <6C269096-47A8-4F23-87AC-512656D4A084@elevated-dev.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox