From: Patrick Mulrooney <mulroony@gmail.com>
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (1.0)
Date: Tue, 18 Dec 2018 23:36:45 -0800
Subject: Re: Increasing parallelism of queries while using file fdw and
 partitions
Message-Id: <D69CA851-AD09-48EE-A51C-13DC2032ED0D@gmail.com>
References: <aea30799-809c-f027-1a4b-0c03c7efa3f6@gmail.com>
 <20181219065149.GF8042@telsasoft.com>
In-Reply-To: <20181219065149.GF8042@telsasoft.com>
To: pgsql-performance@lists.postgresql.org
Precedence: bulk

Justin,

Thanks for the idea. I pulled down the source for multicorn and added that t=
o it. I do not see parallel queries in the analyze output (unless I force it=
 and then it only gets one worker), but it does look like it is reading more=
 than one file at once if I go with a non-partitioned table that looks at al=
l the files. Not any better if I have the table split up into partitions.=20=


So it=E2=80=99s better, but still curious if this would work with partitions=
.=20

Thanks again.=20

Pat

> On Dec 18, 2018, at 22:51, Justin Pryzby <pryzby@telsasoft.com> wrote:
>=20
>> On Tue, Dec 18, 2018 at 08:39:36PM -0800, Patrick Mulrooney wrote:
>> Wondering if anyone had any thoughts on how to tweak my setup to get it t=
o
>> read many files at once instead of one at a time when using file fdw and
>> partitions.
>=20
> I found this:
>=20
> https://www.postgresql.org/docs/current/parallel-safety.html
> |The following operations are always parallel restricted.
> |Scans of foreign tables, unless the foreign data wrapper has an IsForeign=
ScanParallelSafe API which indicates otherwise.
>=20
> https://github.com/ergo70/vcf_fdw/blob/master/vcf_fdw/__init__.py
> =3D> has no such API marker, since it's couple years old, same as multicor=
n.
>=20
> Justin