Subject: Re: logical decoding / rewrite map vs. maxAllocatedDescs
To: Andres Freund <andres@anarazel.de>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>
References: <20180809174746.aa6rbpsu4zjipzif@alvherre.pgsql>
 <b346c3eb-0e65-29ec-47db-0d9a46feaa2c@2ndquadrant.com>
 <20180810211340.nlgyfa7jlmkjcs5m@alap3.anarazel.de>
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Message-ID: <79a0c7f4-89fb-cf8d-82eb-5a7ca8e08d56@2ndquadrant.com>
Date: Sat, 11 Aug 2018 15:43:55 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.5.2
MIME-Version: 1.0
In-Reply-To: <20180810211340.nlgyfa7jlmkjcs5m@alap3.anarazel.de>
Content-Type: multipart/mixed;
 boundary="------------18D063A07D59A581A45628FF"
Content-Language: en-US
Precedence: bulk

This is a multi-part message in MIME format.
--------------18D063A07D59A581A45628FF
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

On 08/10/2018 11:13 PM, Andres Freund wrote:
> On 2018-08-10 22:57:57 +0200, Tomas Vondra wrote:
>>
>>
>> On 08/09/2018 07:47 PM, Alvaro Herrera wrote:
>>> On 2018-Aug-09, Tomas Vondra wrote:
>>>
>>>> I suppose there are reasons why it's done this way, and admittedly the test
>>>> that happens to trigger this is a bit extreme (essentially running pgbench
>>>> concurrently with 'vacuum full pg_class' in a loop). I'm not sure it's
>>>> extreme enough to deem it not an issue, because people using many temporary
>>>> tables often deal with bloat by doing frequent vacuum full on catalogs.
>>>
>>> Actually, it seems to me that ApplyLogicalMappingFile is just leaking
>>> the file descriptor for no good reason.  There's a different
>>> OpenTransientFile call in ReorderBufferRestoreChanges that is not
>>> intended to be closed immediately, but the other one seems a plain bug,
>>> easy enough to fix.
>>>
>>
>> Indeed. Adding a CloseTransientFile to ApplyLogicalMappingFile
>> solves the issue with hitting maxAllocatedDecs. Barring objections
>> I'll commit this shortly.
> 
> Yea, that's clearly a bug. I've not seen a patch, so I can't quite
> formally sign off, but it seems fairly obvious.
> 

I think the fix can be as simple as attached ... I'm mostly afk for the
weekend, so I'll commit & backpatch on Monday or so.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--------------18D063A07D59A581A45628FF
Content-Type: text/x-patch;
 name="reorderbuffer-close.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="reorderbuffer-close.patch"

diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 47e669578f..1a1a0fa469 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -3327,6 +3327,8 @@ ApplyLogicalMappingFile(HTAB *tuplecid_data, Oid relid, const char *fname)
 			new_ent->combocid = ent->combocid;
 		}
 	}
+
+	CloseTransientFile(fd);
 }
 
 

--------------18D063A07D59A581A45628FF--