From: Patrick Mulrooney <mulroony@gmail.com>
To: pgsql-performance@lists.postgresql.org
Subject: Re: Increasing parallelism of queries while using file fdw and partitions
Date: Tue, 18 Dec 2018 23:36:45 -0800
Message-ID: <D69CA851-AD09-48EE-A51C-13DC2032ED0D@gmail.com> (raw)
In-Reply-To: <20181219065149.GF8042@telsasoft.com>
References: <aea30799-809c-f027-1a4b-0c03c7efa3f6@gmail.com>
<20181219065149.GF8042@telsasoft.com>
Justin,
Thanks for the idea. I pulled down the source for multicorn and added that to it. I do not see parallel queries in the analyze output (unless I force it and then it only gets one worker), but it does look like it is reading more than one file at once if I go with a non-partitioned table that looks at all the files. Not any better if I have the table split up into partitions.
So it’s better, but still curious if this would work with partitions.
Thanks again.
Pat
> On Dec 18, 2018, at 22:51, Justin Pryzby <pryzby@telsasoft.com> wrote:
>
>> On Tue, Dec 18, 2018 at 08:39:36PM -0800, Patrick Mulrooney wrote:
>> Wondering if anyone had any thoughts on how to tweak my setup to get it to
>> read many files at once instead of one at a time when using file fdw and
>> partitions.
>
> I found this:
>
> https://www.postgresql.org/docs/current/parallel-safety.html
> |The following operations are always parallel restricted.
> |Scans of foreign tables, unless the foreign data wrapper has an IsForeignScanParallelSafe API which indicates otherwise.
>
> https://github.com/ergo70/vcf_fdw/blob/master/vcf_fdw/__init__.py
> => has no such API marker, since it's couple years old, same as multicorn.
>
> Justin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: mulroony@gmail.com, pgsql-performance@lists.postgresql.org
Subject: Re: Increasing parallelism of queries while using file fdw and partitions
In-Reply-To: <D69CA851-AD09-48EE-A51C-13DC2032ED0D@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox