Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gZVi7-0000NO-Vo for pgsql-performance@arkaria.postgresql.org; Wed, 19 Dec 2018 06:52:04 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1gZVi6-0004b3-I6 for pgsql-performance@arkaria.postgresql.org; Wed, 19 Dec 2018 06:52:02 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gZVi6-0004aw-7M for pgsql-performance@lists.postgresql.org; Wed, 19 Dec 2018 06:52:02 +0000 Received: from mail-io1-xd2b.google.com ([2607:f8b0:4864:20::d2b]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gZVhy-0005E0-5O for pgsql-performance@lists.postgresql.org; Wed, 19 Dec 2018 06:52:01 +0000 Received: by mail-io1-xd2b.google.com with SMTP id k7so14759003iob.6 for ; Tue, 18 Dec 2018 22:51:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telsasoft-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=vSWtxowdyFEbSi/IC0XhtRQ2dTXNcMHQCjynwYVlp+M=; b=PXaABRk9/5yd4c/YFhZG4pWcJTk8ANeuffWe7K0c7f0hNt0Q5U45xpcPPiSTWZ78L0 8RGsnYn4AOcf21kIrpqlyErcwYCNmu5xep3QgwEXEfAxOR1GeaMX+bepgk8TPHr9MyMw HNfg2s8sBGo84ysn8e78hjt98xFpf3Gb4M7F8PuSxOrmrhm5CyOGV9Hg/8Q58k679/s+ ei7bfZBk6NQ0vSdHsDIC4NhoQx04oiueQcC0Jg5AOJCTHUDKdyv4dSbWXYAowF6xNxmq oJb5hYj0GRY/K+HG9d4vv5mf7Lad24fREqGeonkqBG+Guygz/a5Yih0JFjCNz520jjKv P9ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=vSWtxowdyFEbSi/IC0XhtRQ2dTXNcMHQCjynwYVlp+M=; b=jkx2Q9ixDfvrA54GpNKjSHj5oMGVOW4Hh9RqszYNiriHciYfmsrpnWeDzu6S3f2qff VyRY1NJf1/Nn4jykjqNFe/TeOXbpfhQBPBoiua3akyR5XYR7cLr7ys3EAZl/FcWur+5i CrZ8TmCyJwOyayErU4gYtID1oxEbRXmPypQFlKoAS5hvkXh9lADJcLm7hDJQppEyS6Mv eN3ADPHrJod29vGFnUVAJxSpISZpG9bJx4D+y07k+/JojUwHgi+lwcvGykrlaFe+On6F hJt1lQDrlAuDESkQQ+ApwlNQDdqWlJmPVS/zFQc2bak7tgEQ8JCuo6Fk9EckiV9prRgh PBvg== X-Gm-Message-State: AA+aEWbQCqw8sCvhr8GBrxwBaRHGBXJwQvAkwy1c1Jqls+EqfyjCa0N2 xJFlKWQmcEW2N5nOqhdfyXDk8Q== X-Google-Smtp-Source: AFSGD/UcgXZUEFF91fAJVdYJb+cN4i5OHnePIorRXt1Rm0seVlgNYimrpKSpoNFwom4Nk2Ss6N/z0g== X-Received: by 2002:a6b:14c6:: with SMTP id 189mr16713352iou.179.1545202311770; Tue, 18 Dec 2018 22:51:51 -0800 (PST) Received: from pryzbyj (charmander.telsasoft.com. [50.244.222.1]) by smtp.gmail.com with ESMTPSA id c3sm7895501ioi.2.2018.12.18.22.51.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Dec 2018 22:51:51 -0800 (PST) Received: by pryzbyj (Postfix, from userid 1000) id 1B338804E00; Wed, 19 Dec 2018 00:51:49 -0600 (CST) Date: Wed, 19 Dec 2018 00:51:49 -0600 From: Justin Pryzby To: Patrick Mulrooney Cc: pgsql-performance@lists.postgresql.org Subject: Re: Increasing parallelism of queries while using file fdw and partitions Message-ID: <20181219065149.GF8042@telsasoft.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On Tue, Dec 18, 2018 at 08:39:36PM -0800, Patrick Mulrooney wrote: > Wondering if anyone had any thoughts on how to tweak my setup to get it to > read many files at once instead of one at a time when using file fdw and > partitions. I found this: https://www.postgresql.org/docs/current/parallel-safety.html |The following operations are always parallel restricted. |Scans of foreign tables, unless the foreign data wrapper has an IsForeignScanParallelSafe API which indicates otherwise. https://github.com/ergo70/vcf_fdw/blob/master/vcf_fdw/__init__.py => has no such API marker, since it's couple years old, same as multicorn. Justin