pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed

From: Shigeru HANADA <shigeru.hanada@gmail.com>
To: Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
Cc: pgsql-hackers@postgresql.org
Subject: Re: WIP: Collecting statistics on CSV file data
Date: Thu, 05 Apr 2012 21:10:39 +0900
Message-ID: <4F7D8BBF.4050004@gmail.com> (raw)
In-Reply-To: <4EE74C5F.3000309@lab.ntt.co.jp>
References: <4E6DE18B.4010505@lab.ntt.co.jp>
	<4E7861C1.2000607@gmail.com>
	<4E8EC1D8.3080700@lab.ntt.co.jp>
	<4E9C6573.8050507@gmail.com>
	<4E9FF04B.3050504@lab.ntt.co.jp>
	<4EB7C07A.5030205@gmail.com>
	<4EC60883.2050905@lab.ntt.co.jp>
	<4EC648CA.5050904@gmail.com>
	<4EC9CBC5.3070500@lab.ntt.co.jp>
	<4EE1FC38.5080707@lab.ntt.co.jp>
	<4EE5D862.1080809@gmail.com>
	<4EE74C5F.3000309@lab.ntt.co.jp>

(2011/12/13 22:00), Etsuro Fujita wrote:
> Thank you for your effectiveness experiments and proposals for
> improvements.  I updated the patch according to your proposals.
> Attached is the updated version of the patch.

Hi all,

I've revised the v6.1 patch and created v7 patch, though dead line of
this CF is coming closer.  I think that this feature provides a way to
improve plans for foreign tables significantly, so I hope that this
feature is available in 9.2.

I'd like to show overview of the patch again for ease of review.

New FDW API function
====================
This patch adds new FDW API function AnalyzeForeignTable to FdwRoutine
which can be used to support updating local statistics of foreign table.
 This function is invoked when ANALYZE command is executed against a
foreign table explicitly.  This handler function is optional, so if
underlying FDW set NULL for the pointer, PostgreSQL doesn't touch
statistics but emits a message about skipping.

It's not required to FDWs to implement fully featured analyzer by
itself.  They can use core routines, do_analyze_rel and others, for most
difficult part of analyzing.  What FDWs should do is to provide a
sampling function and call do_analyze_rel with passing the sampling
function as argument in their concrete AnalyzeForeignTable.

AnalyzeForeignTable (or sampling function) can report FDW-specific
additional information by calling ereport() with given elevel.

Once we've considered an idea that FDW stores statistics information
without calling do_analyze_rel, but it seems very hard to implement, and
not so efficient.  I tried to implement such handler in pgsql_fdw (which
seeems easiest to achieve) by getting remote statistics by "SELECT *
FROM pg_statistics", but it has several issues such as:

1) Highly privileged user on remote side should be mapped to ANALYZE invoker
2) Structure and semantics might be difficult on remote side, if
versions are not same.
3) We need to convert anyarray to anyarray through text representation.
 We know type of elements, but bothering works are needed.

DDL changes
===========
ALTER FOREIGN TABLE supports SET STATISTICS clause and n_distinct
option.  Former changes per-attribute statistics target, and latter
overrides calculated statistics.  n_distinct_inherited is not available
because foreign tables can't be inherited.

psql support
============
psql completes foreign table names after the keyword "ANALYZE" in
addition to ordinary tables.  Of course newly added statistics target is
shown in \d+ command.

file_fdw
========
This patch contains a use case of new handler function in
contrib/file_fdw.  Since file_fdw reads data from a flat file,
fileAnalyzeForeignTable uses similar algorithm to ordinary tables;  it
samples first N rows first, and replaces them randomly with subsequent
rows.  Also file_fdw updates pg_class.relpages by calculating number of
pages from size of the data file.

To allow FDWs to implement sampling argorighm like this, several
functions are exported from analyze.c, e.g. random_fract,
init_selection_state, and get_next_S.

pgsql_fdw
=========
Though it's not fully finished, I've implemented ANALYZE handler for
pgsql_fdw.  Please extract pgsql_fdw.tar.gz into contrib (or use pgxs)
to try it.

Regards,
-- 
Shigeru HANADA
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index e890770..f84a01f 100644
*** a/contrib/file_fdw/file_fdw.c
--- b/contrib/file_fdw/file_fdw.c
***************
*** 20,25 ****
--- 20,26 ----
  #include "commands/copy.h"
  #include "commands/defrem.h"
  #include "commands/explain.h"
+ #include "commands/vacuum.h"
  #include "foreign/fdwapi.h"
  #include "foreign/foreign.h"
  #include "miscadmin.h"
*************** static void fileBeginForeignScan(Foreign
*** 123,128 ****
--- 124,132 ----
  static TupleTableSlot *fileIterateForeignScan(ForeignScanState *node);
  static void fileReScanForeignScan(ForeignScanState *node);
  static void fileEndForeignScan(ForeignScanState *node);
+ static void fileAnalyzeForeignTable(Relation onerel,
+ 					VacuumStmt *vacstmt,
+ 					int elevel);
  
  /*
   * Helper functions
*************** static void estimate_size(PlannerInfo *r
*** 136,142 ****
  static void estimate_costs(PlannerInfo *root, RelOptInfo *baserel,
  			   FileFdwPlanState *fdw_private,
  			   Cost *startup_cost, Cost *total_cost);
! 
  
  /*
   * Foreign-data wrapper handler function: return a struct with pointers
--- 140,149 ----
  static void estimate_costs(PlannerInfo *root, RelOptInfo *baserel,
  			   FileFdwPlanState *fdw_private,
  			   Cost *startup_cost, Cost *total_cost);
! static int	acquire_sample_rows(Relation onerel,
! 			   HeapTuple *rows, int targrows,
! 			   double *totalrows, double *totaldeadrows,
! 			   BlockNumber *totalpages, int elevel);
  
  /*
   * Foreign-data wrapper handler function: return a struct with pointers
*************** file_fdw_handler(PG_FUNCTION_ARGS)
*** 155,160 ****
--- 162,168 ----
  	fdwroutine->IterateForeignScan = fileIterateForeignScan;
  	fdwroutine->ReScanForeignScan = fileReScanForeignScan;
  	fdwroutine->EndForeignScan = fileEndForeignScan;
+ 	fdwroutine->AnalyzeForeignTable = fileAnalyzeForeignTable;
  
  	PG_RETURN_POINTER(fdwroutine);
  }
*************** estimate_size(PlannerInfo *root, RelOptI
*** 662,693 ****
  	double		nrows;
  
  	/*
! 	 * Get size of the file.  It might not be there at plan time, though, in
! 	 * which case we have to use a default estimate.
  	 */
! 	if (stat(fdw_private->filename, &stat_buf) < 0)
! 		stat_buf.st_size = 10 * BLCKSZ;
  
! 	/*
! 	 * Convert size to pages for use in I/O cost estimate later.
! 	 */
! 	pages = (stat_buf.st_size + (BLCKSZ - 1)) / BLCKSZ;
! 	if (pages < 1)
! 		pages = 1;
  
! 	fdw_private->pages = pages;
  
! 	/*
! 	 * Estimate the number of tuples in the file.  We back into this estimate
! 	 * using the planner's idea of the relation width; which is bogus if not
! 	 * all columns are being read, not to mention that the text representation
! 	 * of a row probably isn't the same size as its internal representation.
! 	 * FIXME later.
! 	 */
! 	tuple_width = MAXALIGN(baserel->width) + MAXALIGN(sizeof(HeapTupleHeaderData));
  
! 	ntuples = clamp_row_est((double) stat_buf.st_size / (double) tuple_width);
  
  	fdw_private->ntuples = ntuples;
  
  	/*
--- 670,716 ----
  	double		nrows;
  
  	/*
! 	 * Use statistics stored in pg_class as is if any.  Otherwise, calculate
! 	 * them from file size and average tuple width.
  	 */
! 	if (baserel->pages > 0)
! 	{
! 		pages = baserel->pages;
! 		ntuples = baserel->tuples;
! 	}
! 	else
! 	{
! 		/*
! 		 * Get size of the file.  It might not be there at plan time, though,
! 		 * in which case we have to use a default estimate.
! 		 */
! 		if (stat(fdw_private->filename, &stat_buf) < 0)
! 			stat_buf.st_size = 10 * BLCKSZ;
  
! 		/*
! 		 * Convert size to pages for use in I/O cost estimate later.
! 		 */
! 		pages = (stat_buf.st_size + (BLCKSZ - 1)) / BLCKSZ;
! 		if (pages < 1)
! 			pages = 1;
  
! 		/*
! 		 * Estimate the number of tuples in the file.  We back into this
! 		 * estimate using the planner's idea of the relation width; which is
! 		 * bogus if not all columns are being read, not to mention that the
! 		 * text representation of a row probably isn't the same size as its
! 		 * internal representation.  FIXME later.
! 		 */
! 		tuple_width = MAXALIGN(baserel->width) +
! 					  MAXALIGN(sizeof(HeapTupleHeaderData));
  
! 		ntuples = clamp_row_est((double) stat_buf.st_size /
! 								(double) tuple_width);
  
! 	}
  
+ 	/* Pass estimates to subsequent functions via FileFdwPlanState. */
+ 	fdw_private->pages = pages;
  	fdw_private->ntuples = ntuples;
  
  	/*
*************** estimate_size(PlannerInfo *root, RelOptI
*** 709,714 ****
--- 732,747 ----
  }
  
  /*
+  * fileAnalyzeForeignTable
+  *		Analyze foreign table
+  */
+ static void
+ fileAnalyzeForeignTable(Relation onerel, VacuumStmt *vacstmt, int elevel)
+ {
+ 	do_analyze_rel(onerel, vacstmt, elevel, false, acquire_sample_rows);
+ }
+ 
+ /*
   * Estimate costs of scanning a foreign table.
   *
   * Results are returned in *startup_cost and *total_cost.
*************** estimate_costs(PlannerInfo *root, RelOpt
*** 736,738 ****
--- 769,957 ----
  	run_cost += cpu_per_tuple * ntuples;
  	*total_cost = *startup_cost + run_cost;
  }
+ 
+ /*
+  * acquire_sample_rows -- acquire a random sample of rows from the table
+  *
+  * Selected rows are returned in the caller-allocated array rows[], which must
+  * have at least targrows entries.  The actual number of rows selected is 
+  * returned as the function result.  We also count the number of valid rows in
+  * the table, and return it into *totalrows.
+  *
+  * The returned list of tuples is in order by physical position in the table.
+  * (We will rely on this later to derive correlation estimates.)
+  */
+ static int
+ acquire_sample_rows(Relation onerel, HeapTuple *rows, int targrows,
+ 					double *totalrows, double *totaldeadrows,
+ 					BlockNumber *totalpages, int elevel)
+ {
+ 	int			numrows = 0;
+ 	int			invalrows = 0; /* total # rows violating
+ 								  the NOT NULL constraints */
+ 	double		validrows = 0; /* total # rows collected */
+ 	double		rowstoskip = -1; /* -1 means not set yet */
+ 	double		rstate;
+ 	HeapTuple	tuple;
+ 	TupleDesc	tupDesc;
+ 	TupleConstr *constr;
+ 	int			natts;
+ 	int			attrChk;
+ 	Datum	   *values;
+ 	bool	   *nulls;
+ 	bool		found;
+ 	bool		sample_it = false;
+ 	char	   *filename;
+ 	struct stat	stat_buf;
+ 	List	   *options;
+ 	CopyState	cstate;
+ 	ErrorContextCallback errcontext;
+ 
+ 	Assert(onerel);
+ 	Assert(targrows > 0);
+ 
+ 	tupDesc = RelationGetDescr(onerel);
+ 	constr = tupDesc->constr;
+ 	natts = tupDesc->natts;
+ 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
+ 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
+ 
+ 	/* Fetch options of foreign table */
+ 	fileGetOptions(RelationGetRelid(onerel), &filename, &options);
+ 
+ 	/*
+ 	 * Get size of the file.
+ 	 */
+ 	if (stat(filename, &stat_buf) < 0)
+ 		ereport(ERROR,
+ 				(errcode_for_file_access(),
+ 				 errmsg("could not stat file \"%s\": %m",
+ 						filename)));
+ 
+ 	/*
+ 	 * Convert size to pages for use in I/O cost estimate.
+ 	 */
+ 	*totalpages = (stat_buf.st_size + (BLCKSZ - 1)) / BLCKSZ;
+ 	if (*totalpages < 1)
+ 		*totalpages = 1;
+ 
+ 	/*
+ 	 * Create CopyState from FDW options.  We always acquire all columns, so
+ 	 * as to match the expected ScanTupleSlot signature.
+ 	 */
+ 	cstate = BeginCopyFrom(onerel, filename, NIL, options);
+ 
+ 	/* Prepare for sampling rows */
+ 	rstate = init_selection_state(targrows);
+ 
+ 	/* Set up callback to identify error line number. */
+ 	errcontext.callback = CopyFromErrorCallback;
+ 	errcontext.arg = (void *) cstate;
+ 	errcontext.previous = error_context_stack;
+ 	error_context_stack = &errcontext;
+ 
+ 	for (;;)
+ 	{
+ 		sample_it = true;
+ 
+ 		/*
+ 		 * Check for user-requested abort.
+ 		 */
+ 		CHECK_FOR_INTERRUPTS();
+ 
+ 		found = NextCopyFrom(cstate, NULL, values, nulls, NULL);
+ 
+ 		if (!found)
+ 			break;
+ 
+ 		tuple = heap_form_tuple(tupDesc, values, nulls);
+ 
+ 		if (constr && constr->has_not_null)
+ 		{
+ 			for (attrChk = 1; attrChk <= natts; attrChk++)
+ 			{
+ 				if (onerel->rd_att->attrs[attrChk - 1]->attnotnull &&
+ 					heap_attisnull(tuple, attrChk))
+ 				{
+ 					sample_it = false;
+ 					break;
+ 				}
+ 			}
+ 		}
+ 
+ 		if (!sample_it)
+ 		{
+ 			invalrows += 1;
+ 			heap_freetuple(tuple);
+ 			continue;
+ 		}
+ 
+ 		/*
+ 		 * The first targrows sample rows are simply copied into the
+ 		 * reservoir. Then we start replacing tuples in the sample
+ 		 * until we reach the end of the relation.	This algorithm is
+ 		 * from Jeff Vitter's paper (see full citation below). It
+ 		 * works by repeatedly computing the number of tuples to skip
+ 		 * before selecting a tuple, which replaces a randomly chosen
+ 		 * element of the reservoir (current set of tuples).  At all
+ 		 * times the reservoir is a true random sample of the tuples
+ 		 * we've passed over so far, so when we fall off the end of
+ 		 * the relation we're done.
+ 		 */
+ 		if (numrows < targrows)
+ 			rows[numrows++] = heap_copytuple(tuple);
+ 		else
+ 		{
+ 			/*
+ 			 * t in Vitter's paper is the number of records already
+ 			 * processed.  If we need to compute a new S value, we
+ 			 * must use the not-yet-incremented value of samplerows as
+ 			 * t.
+ 			 */
+ 			if (rowstoskip < 0)
+ 				rowstoskip = get_next_S(validrows, targrows, &rstate);
+ 
+ 			if (rowstoskip <= 0)
+ 			{
+ 				/*
+ 				 * Found a suitable tuple, so save it, replacing one
+ 				 * old tuple at random
+ 				 */
+ 				int k = (int) (targrows * random_fract());
+ 
+ 				Assert(k >= 0 && k < targrows);
+ 				heap_freetuple(rows[k]);
+ 				rows[k] = heap_copytuple(tuple);
+ 			}
+ 
+ 			rowstoskip -= 1;
+ 		}
+ 
+ 		validrows += 1;
+ 		heap_freetuple(tuple);
+ 	}
+ 
+ 	/* Remove error callback. */
+ 	error_context_stack = errcontext.previous;
+ 
+ 	*totalrows = validrows;
+ 	*totaldeadrows = 0;
+ 
+ 	EndCopyFrom(cstate);
+ 
+ 	pfree(values);
+ 	pfree(nulls);
+ 
+ 	/*
+ 	 * Emit some interesting relation info
+ 	 */
+ 	ereport(elevel,
+ 			(errmsg("\"%s\": scanned, "
+ 					"containing %d valid rows and %d invalid rows; "
+ 					"%d rows in sample, %d total rows",
+ 					RelationGetRelationName(onerel),
+ 					(int) validrows, invalrows,
+ 					numrows, (int) *totalrows)));
+ 
+ 	return numrows;
+ }
diff --git a/contrib/file_fdw/input/file_fdw.source b/contrib/file_fdw/input/file_fdw.source
index 8e3d553..fddd9cd 100644
*** a/contrib/file_fdw/input/file_fdw.source
--- b/contrib/file_fdw/input/file_fdw.source
*************** EXECUTE st(100);
*** 111,116 ****
--- 111,121 ----
  EXECUTE st(100);
  DEALLOCATE st;
  
+ -- statistics collection tests
+ ANALYZE agg_csv;
+ SELECT relpages, reltuples FROM pg_class WHERE relname = 'agg_csv';
+ SELECT * FROM pg_stats WHERE tablename = 'agg_csv' ORDER BY attname;
+ 
  -- tableoid
  SELECT tableoid::regclass, b FROM agg_csv;
  
diff --git a/contrib/file_fdw/output/file_fdw.source b/contrib/file_fdw/output/file_fdw.source
index 84f0750..5be5fe0 100644
*** a/contrib/file_fdw/output/file_fdw.source
--- b/contrib/file_fdw/output/file_fdw.source
*************** EXECUTE st(100);
*** 174,179 ****
--- 174,194 ----
  (1 row)
  
  DEALLOCATE st;
+ -- statistics collection tests
+ ANALYZE agg_csv;
+ SELECT relpages, reltuples FROM pg_class WHERE relname = 'agg_csv';
+  relpages | reltuples 
+ ----------+-----------
+         1 |         3
+ (1 row)
+ 
+ SELECT * FROM pg_stats WHERE tablename = 'agg_csv' ORDER BY attname;
+  schemaname | tablename | attname | inherited | null_frac | avg_width | n_distinct | most_common_vals | most_common_freqs |    histogram_bounds     | correlation | most_common_elems | most_common_elem_freqs | elem_count_histogram 
+ ------------+-----------+---------+-----------+-----------+-----------+------------+------------------+-------------------+-------------------------+-------------+-------------------+------------------------+----------------------
+  public     | agg_csv   | a       | f         |         0 |         2 |         -1 |                  |                   | {0,42,100}              |        -0.5 |                   |                        | 
+  public     | agg_csv   | b       | f         |         0 |         4 |         -1 |                  |                   | {0.09561,99.097,324.78} |         0.5 |                   |                        | 
+ (2 rows)
+ 
  -- tableoid
  SELECT tableoid::regclass, b FROM agg_csv;
   tableoid |    b    
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index f7bf3d8..4f962b7 100644
*** a/doc/src/sgml/fdwhandler.sgml
--- b/doc/src/sgml/fdwhandler.sgml
*************** EndForeignScan (ForeignScanState *node);
*** 277,282 ****
--- 277,305 ----
      </para>
  
      <para>
+ <programlisting>
+ void
+ AnalyzeForeignTable (Relation onerel,
+                      VacuumStmt *vacstmt,
+                      int elevel);
+ </programlisting>
+ 
+      Collect statistics on a foreign table and store the results in the
+      pg_class and pg_statistics system catalogs.
+      This is optional, and if implemented, called when <command>ANALYZE</>
+      command is run.  The statistics are used by the query planner in order to
+      make good choices of query plans.
+     </para>
+ 
+     <para>
+      The function can be implemented by writing a sampling function that
+      acquires a random sample of rows from an external data source and
+      then by calling <function>do_analyze_rel</>, where you should pass
+      the sampling function as an argument.
+      The function must be set to NULL if it isn't implemented.
+     </para>
+ 
+     <para>
       The <structname>FdwRoutine</> struct type is declared in
       <filename>src/include/foreign/fdwapi.h</>, which see for additional
       details.
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 93c3ff5..54d0838 100644
*** a/doc/src/sgml/maintenance.sgml
--- b/doc/src/sgml/maintenance.sgml
***************
*** 284,289 ****
--- 284,293 ----
      <command>ANALYZE</> strictly as a function of the number of rows
      inserted or updated; it has no knowledge of whether that will lead
      to meaningful statistical changes.
+     Note that the autovacuum daemon does not issue <command>ANALYZE</>
+     commands on foreign tables.  It is recommended to run manually-managed
+     <command>ANALYZE</> commands as needed, which typically are executed
+     according to a schedule by cron or Task Scheduler scripts.
     </para>
  
     <para>
diff --git a/doc/src/sgml/ref/alter_foreign_table.sgml b/doc/src/sgml/ref/alter_foreign_table.sgml
index c4cdaa8..af5c0a8 100644
*** a/doc/src/sgml/ref/alter_foreign_table.sgml
--- b/doc/src/sgml/ref/alter_foreign_table.sgml
*************** ALTER FOREIGN TABLE [ IF EXISTS ] <repla
*** 36,41 ****
--- 36,44 ----
      DROP [ COLUMN ] [ IF EXISTS ] <replaceable class="PARAMETER">column</replaceable> [ RESTRICT | CASCADE ]
      ALTER [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> [ SET DATA ] TYPE <replaceable class="PARAMETER">type</replaceable>
      ALTER [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> { SET | DROP } NOT NULL
+     ALTER [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> SET STATISTICS <replaceable class="PARAMETER">integer</replaceable>
+     ALTER [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> SET ( <replaceable class="PARAMETER">attribute_option</replaceable> = <replaceable class="PARAMETER">value</replaceable> [, ... ] )
+     ALTER [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> RESET ( <replaceable class="PARAMETER">attribute_option</replaceable> [, ... ] )
      ALTER [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> OPTIONS ( [ ADD | SET | DROP ] <replaceable class="PARAMETER">option</replaceable> ['<replaceable class="PARAMETER">value</replaceable>'] [, ... ])
      OWNER TO <replaceable class="PARAMETER">new_owner</replaceable>
      OPTIONS ( [ ADD | SET | DROP ] <replaceable class="PARAMETER">option</replaceable> ['<replaceable class="PARAMETER">value</replaceable>'] [, ... ])
*************** ALTER FOREIGN TABLE [ IF EXISTS ] <repla
*** 104,109 ****
--- 107,156 ----
     </varlistentry>
  
     <varlistentry>
+     <term><literal>SET STATISTICS</literal></term>
+     <listitem>
+      <para>
+       This form
+       sets the per-column statistics-gathering target for subsequent
+       <xref linkend="sql-analyze"> operations.
+       The target can be set in the range 0 to 10000; alternatively, set it
+       to -1 to revert to using the system default statistics
+       target (<xref linkend="guc-default-statistics-target">).
+      </para>
+     </listitem>
+    </varlistentry>
+ 
+    <varlistentry>
+     <term><literal>SET ( <replaceable class="PARAMETER">attribute_option</replaceable> = <replaceable class="PARAMETER">value</replaceable> [, ... ] )</literal></term>
+     <term><literal>RESET ( <replaceable class="PARAMETER">attribute_option</replaceable> [, ... ] )</literal></term>
+     <listitem>
+      <para>
+       This form
+       sets or resets a per-attribute option.  Currently, the only defined
+       per-attribute option is <literal>n_distinct</>, which overrides
+       the number-of-distinct-values estimates made by subsequent
+       <xref linkend="sql-analyze"> operations. 
+       When set to a positive value, <command>ANALYZE</> will assume that
+       the column contains exactly the specified number of distinct nonnull
+       values.
+       When set to a negative value, which must be greater than or equal
+       to -1, <command>ANALYZE</> will assume that the number of distinct
+       nonnull values in the column is linear in the size of the foreign
+       table; the exact count is to be computed by multiplying the estimated
+       foreign table size by the absolute value of the given number.
+       For example,
+       a value of -1 implies that all values in the column are distinct,
+       while a value of -0.5 implies that each value appears twice on the
+       average.
+       This can be useful when the size of the foreign table changes over
+       time, since the multiplication by the number of rows in the foreign
+       table is not performed until query planning time.  Specify a value
+       of 0 to revert to estimating the number of distinct values normally.
+      </para>
+     </listitem>
+    </varlistentry>
+ 
+    <varlistentry>
      <term><literal>OWNER</literal></term>
      <listitem>
       <para>
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 8c9057b..97e9ad6 100644
*** a/doc/src/sgml/ref/analyze.sgml
--- b/doc/src/sgml/ref/analyze.sgml
*************** ANALYZE [ VERBOSE ] [ <replaceable class
*** 39,47 ****
  
    <para>
     With no parameter, <command>ANALYZE</command> examines every table in the
!    current database.  With a parameter, <command>ANALYZE</command> examines
!    only that table.  It is further possible to give a list of column names,
!    in which case only the statistics for those columns are collected.
    </para>
   </refsect1>
  
--- 39,49 ----
  
    <para>
     With no parameter, <command>ANALYZE</command> examines every table in the
!    current database except for foreign tables.  With a parameter,
!    <command>ANALYZE</command> examines only that table.  For a foreign table,
!    it is necessary to specify the name of that table.  It is further possible
!    to give a list of column names, in which case only the statistics for those
!    columns are collected.
    </para>
   </refsect1>
  
*************** ANALYZE [ VERBOSE ] [ <replaceable class
*** 63,69 ****
      <listitem>
       <para>
        The name (possibly schema-qualified) of a specific table to
!       analyze. Defaults to all tables in the current database.
       </para>
      </listitem>
     </varlistentry>
--- 65,72 ----
      <listitem>
       <para>
        The name (possibly schema-qualified) of a specific table to
!       analyze. Defaults to all tables in the current database except
!       for foreign tables.
       </para>
      </listitem>
     </varlistentry>
*************** ANALYZE [ VERBOSE ] [ <replaceable class
*** 137,143 ****
     In rare situations, this non-determinism will cause the planner's
     choices of query plans to change after <command>ANALYZE</command> is run.
     To avoid this, raise the amount of statistics collected by
!    <command>ANALYZE</command>, as described below.
    </para>
  
    <para>
--- 140,148 ----
     In rare situations, this non-determinism will cause the planner's
     choices of query plans to change after <command>ANALYZE</command> is run.
     To avoid this, raise the amount of statistics collected by
!    <command>ANALYZE</command>, as described below.  Note that the time
!    needed to analyze on foreign tables depends on the implementation of
!    the foreign data wrapper via which such tables are attached.
    </para>
  
    <para>
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 9cd6e67..6ac9aee 100644
*** a/src/backend/commands/analyze.c
--- b/src/backend/commands/analyze.c
***************
*** 23,28 ****
--- 23,29 ----
  #include "access/xact.h"
  #include "catalog/index.h"
  #include "catalog/indexing.h"
+ #include "catalog/pg_class.h"
  #include "catalog/pg_collation.h"
  #include "catalog/pg_inherits_fn.h"
  #include "catalog/pg_namespace.h"
***************
*** 30,35 ****
--- 31,38 ----
  #include "commands/tablecmds.h"
  #include "commands/vacuum.h"
  #include "executor/executor.h"
+ #include "foreign/foreign.h"
+ #include "foreign/fdwapi.h"
  #include "miscadmin.h"
  #include "nodes/nodeFuncs.h"
  #include "parser/parse_oper.h"
*************** typedef struct AnlIndexData
*** 78,91 ****
  int			default_statistics_target = 100;
  
  /* A few variables that don't seem worth passing around as parameters */
- static int	elevel = -1;
- 
  static MemoryContext anl_context = NULL;
  
  static BufferAccessStrategy vac_strategy;
  
  
- static void do_analyze_rel(Relation onerel, VacuumStmt *vacstmt, bool inh);
  static void BlockSampler_Init(BlockSampler bs, BlockNumber nblocks,
  				  int samplesize);
  static bool BlockSampler_HasMore(BlockSampler bs);
--- 81,91 ----
*************** static void compute_index_stats(Relation
*** 97,110 ****
  static VacAttrStats *examine_attribute(Relation onerel, int attnum,
  				  Node *index_expr);
  static int acquire_sample_rows(Relation onerel, HeapTuple *rows,
! 					int targrows, double *totalrows, double *totaldeadrows);
! static double random_fract(void);
! static double init_selection_state(int n);
! static double get_next_S(double t, int n, double *stateptr);
  static int	compare_rows(const void *a, const void *b);
  static int acquire_inherited_sample_rows(Relation onerel,
  							  HeapTuple *rows, int targrows,
! 							  double *totalrows, double *totaldeadrows);
  static void update_attstats(Oid relid, bool inh,
  				int natts, VacAttrStats **vacattrstats);
  static Datum std_fetch_func(VacAttrStatsP stats, int rownum, bool *isNull);
--- 97,109 ----
  static VacAttrStats *examine_attribute(Relation onerel, int attnum,
  				  Node *index_expr);
  static int acquire_sample_rows(Relation onerel, HeapTuple *rows,
! 					int targrows, double *totalrows, double *totaldeadrows,
! 					BlockNumber *totalpages, int elevel);
  static int	compare_rows(const void *a, const void *b);
  static int acquire_inherited_sample_rows(Relation onerel,
  							  HeapTuple *rows, int targrows,
! 							  double *totalrows, double *totaldeadrows,
! 							  BlockNumber *totalpages, int elevel);
  static void update_attstats(Oid relid, bool inh,
  				int natts, VacAttrStats **vacattrstats);
  static Datum std_fetch_func(VacAttrStatsP stats, int rownum, bool *isNull);
*************** void
*** 118,123 ****
--- 117,124 ----
  analyze_rel(Oid relid, VacuumStmt *vacstmt, BufferAccessStrategy bstrategy)
  {
  	Relation	onerel;
+ 	int			elevel;
+ 	FdwRoutine *fdwroutine;
  
  	/* Set up static variables */
  	if (vacstmt->options & VACOPT_VERBOSE)
*************** analyze_rel(Oid relid, VacuumStmt *vacst
*** 182,191 ****
  	}
  
  	/*
! 	 * Check that it's a plain table; we used to do this in get_rel_oids() but
! 	 * seems safer to check after we've locked the relation.
  	 */
! 	if (onerel->rd_rel->relkind != RELKIND_RELATION)
  	{
  		/* No need for a WARNING if we already complained during VACUUM */
  		if (!(vacstmt->options & VACOPT_VACUUM))
--- 183,194 ----
  	}
  
  	/*
! 	 * Check that it's a plain table or foreign table; we used to do this
! 	 * in get_rel_oids() but seems safer to check after we've locked the
! 	 * relation.
  	 */
! 	if (onerel->rd_rel->relkind != RELKIND_RELATION &&
! 		onerel->rd_rel->relkind != RELKIND_FOREIGN_TABLE)
  	{
  		/* No need for a WARNING if we already complained during VACUUM */
  		if (!(vacstmt->options & VACOPT_VACUUM))
*************** analyze_rel(Oid relid, VacuumStmt *vacst
*** 209,215 ****
  	}
  
  	/*
! 	 * We can ANALYZE any table except pg_statistic. See update_attstats
  	 */
  	if (RelationGetRelid(onerel) == StatisticRelationId)
  	{
--- 212,220 ----
  	}
  
  	/*
! 	 * We can ANALYZE any table except pg_statistic.  See update_attstats.
! 	 * In addition, we can ANALYZE foreign tables if AnalyzeForeignTable
! 	 * callback routines of underlying foreign-data wrappers are implemented.
  	 */
  	if (RelationGetRelid(onerel) == StatisticRelationId)
  	{
*************** analyze_rel(Oid relid, VacuumStmt *vacst
*** 217,222 ****
--- 222,241 ----
  		return;
  	}
  
+ 	if (onerel->rd_rel->relkind == RELKIND_FOREIGN_TABLE)
+ 	{
+  		fdwroutine = GetFdwRoutineByRelId(RelationGetRelid(onerel));
+ 
+ 		if (fdwroutine->AnalyzeForeignTable == NULL)
+ 		{
+ 			ereport(WARNING,
+ 					(errmsg("skipping \"%s\" --- underlying foreign-data wrapper cannot analyze it",
+ 							RelationGetRelationName(onerel))));
+ 			relation_close(onerel, ShareUpdateExclusiveLock);
+ 			return;
+ 		}
+ 	}
+ 
  	/*
  	 * OK, let's do it.  First let other backends know I'm in ANALYZE.
  	 */
*************** analyze_rel(Oid relid, VacuumStmt *vacst
*** 224,239 ****
  	MyPgXact->vacuumFlags |= PROC_IN_ANALYZE;
  	LWLockRelease(ProcArrayLock);
  
! 	/*
! 	 * Do the normal non-recursive ANALYZE.
! 	 */
! 	do_analyze_rel(onerel, vacstmt, false);
  
! 	/*
! 	 * If there are child tables, do recursive ANALYZE.
! 	 */
! 	if (onerel->rd_rel->relhassubclass)
! 		do_analyze_rel(onerel, vacstmt, true);
  
  	/*
  	 * Close source relation now, but keep lock so that no one deletes it
--- 243,280 ----
  	MyPgXact->vacuumFlags |= PROC_IN_ANALYZE;
  	LWLockRelease(ProcArrayLock);
  
! 	if (onerel->rd_rel->relkind == RELKIND_FOREIGN_TABLE)
! 	{
! 		ereport(elevel,
! 				(errmsg("analyzing \"%s.%s\"",
! 						get_namespace_name(RelationGetNamespace(onerel)),
! 						RelationGetRelationName(onerel))));
! 		fdwroutine->AnalyzeForeignTable(onerel, vacstmt, elevel);
! 	}
! 	else
! 	{
! 		/*
! 		 * Do the normal non-recursive ANALYZE.
! 		 */
! 		ereport(elevel,
! 				(errmsg("analyzing \"%s.%s\"",
! 						get_namespace_name(RelationGetNamespace(onerel)),
! 						RelationGetRelationName(onerel))));
! 		do_analyze_rel(onerel, vacstmt, elevel, false, acquire_sample_rows);
  
! 		/*
! 		 * If there are child tables, do recursive ANALYZE.
! 		 */
! 		if (onerel->rd_rel->relhassubclass)
! 		{
! 			ereport(elevel,
! 					(errmsg("analyzing \"%s.%s\" inheritance tree",
! 							get_namespace_name(RelationGetNamespace(onerel)),
! 							RelationGetRelationName(onerel))));
! 			do_analyze_rel(onerel, vacstmt, elevel, true,
! 						   acquire_inherited_sample_rows);
! 		}
! 	}
  
  	/*
  	 * Close source relation now, but keep lock so that no one deletes it
*************** analyze_rel(Oid relid, VacuumStmt *vacst
*** 255,262 ****
  /*
   *	do_analyze_rel() -- analyze one relation, recursively or not
   */
! static void
! do_analyze_rel(Relation onerel, VacuumStmt *vacstmt, bool inh)
  {
  	int			attr_cnt,
  				tcnt,
--- 296,304 ----
  /*
   *	do_analyze_rel() -- analyze one relation, recursively or not
   */
! void
! do_analyze_rel(Relation onerel, VacuumStmt *vacstmt, int elevel, 
! 			   bool inh, SampleRowAcquireFunc acquirefunc)
  {
  	int			attr_cnt,
  				tcnt,
*************** do_analyze_rel(Relation onerel, VacuumSt
*** 271,276 ****
--- 313,319 ----
  				numrows;
  	double		totalrows,
  				totaldeadrows;
+ 	BlockNumber totalpages;
  	HeapTuple  *rows;
  	PGRUsage	ru0;
  	TimestampTz starttime = 0;
*************** do_analyze_rel(Relation onerel, VacuumSt
*** 279,295 ****
  	int			save_sec_context;
  	int			save_nestlevel;
  
- 	if (inh)
- 		ereport(elevel,
- 				(errmsg("analyzing \"%s.%s\" inheritance tree",
- 						get_namespace_name(RelationGetNamespace(onerel)),
- 						RelationGetRelationName(onerel))));
- 	else
- 		ereport(elevel,
- 				(errmsg("analyzing \"%s.%s\"",
- 						get_namespace_name(RelationGetNamespace(onerel)),
- 						RelationGetRelationName(onerel))));
- 
  	/*
  	 * Set up a working context so that we can easily free whatever junk gets
  	 * created.
--- 322,327 ----
*************** do_analyze_rel(Relation onerel, VacuumSt
*** 447,457 ****
  	 */
  	rows = (HeapTuple *) palloc(targrows * sizeof(HeapTuple));
  	if (inh)
! 		numrows = acquire_inherited_sample_rows(onerel, rows, targrows,
! 												&totalrows, &totaldeadrows);
  	else
! 		numrows = acquire_sample_rows(onerel, rows, targrows,
! 									  &totalrows, &totaldeadrows);
  
  	/*
  	 * Compute the statistics.	Temporary results during the calculations for
--- 479,491 ----
  	 */
  	rows = (HeapTuple *) palloc(targrows * sizeof(HeapTuple));
  	if (inh)
! 		numrows = acquirefunc(onerel, rows, targrows,
! 							  &totalrows, &totaldeadrows,
! 							  NULL, elevel);
  	else
! 		numrows = acquirefunc(onerel, rows, targrows,
! 							  &totalrows, &totaldeadrows,
! 							  &totalpages, elevel);
  
  	/*
  	 * Compute the statistics.	Temporary results during the calculations for
*************** do_analyze_rel(Relation onerel, VacuumSt
*** 532,538 ****
  	 */
  	if (!inh)
  		vac_update_relstats(onerel,
! 							RelationGetNumberOfBlocks(onerel),
  							totalrows,
  							visibilitymap_count(onerel),
  							hasindex,
--- 566,572 ----
  	 */
  	if (!inh)
  		vac_update_relstats(onerel,
! 							totalpages,
  							totalrows,
  							visibilitymap_count(onerel),
  							hasindex,
*************** BlockSampler_Next(BlockSampler bs)
*** 1015,1021 ****
   */
  static int
  acquire_sample_rows(Relation onerel, HeapTuple *rows, int targrows,
! 					double *totalrows, double *totaldeadrows)
  {
  	int			numrows = 0;	/* # rows now in reservoir */
  	double		samplerows = 0; /* total # rows collected */
--- 1049,1056 ----
   */
  static int
  acquire_sample_rows(Relation onerel, HeapTuple *rows, int targrows,
! 					double *totalrows, double *totaldeadrows,
! 					BlockNumber *totalpages, int elevel)
  {
  	int			numrows = 0;	/* # rows now in reservoir */
  	double		samplerows = 0; /* total # rows collected */
*************** acquire_sample_rows(Relation onerel, Hea
*** 1030,1035 ****
--- 1065,1072 ----
  	Assert(targrows > 0);
  
  	totalblocks = RelationGetNumberOfBlocks(onerel);
+ 	if (totalpages)
+ 		*totalpages = totalblocks;
  
  	/* Need a cutoff xmin for HeapTupleSatisfiesVacuum */
  	OldestXmin = GetOldestXmin(onerel->rd_rel->relisshared, true);
*************** acquire_sample_rows(Relation onerel, Hea
*** 1252,1258 ****
  }
  
  /* Select a random value R uniformly distributed in (0 - 1) */
! static double
  random_fract(void)
  {
  	return ((double) random() + 1) / ((double) MAX_RANDOM_VALUE + 2);
--- 1289,1295 ----
  }
  
  /* Select a random value R uniformly distributed in (0 - 1) */
! double
  random_fract(void)
  {
  	return ((double) random() + 1) / ((double) MAX_RANDOM_VALUE + 2);
*************** random_fract(void)
*** 1272,1285 ****
   * determines the number of records to skip before the next record is
   * processed.
   */
! static double
  init_selection_state(int n)
  {
  	/* Initial value of W (for use when Algorithm Z is first applied) */
  	return exp(-log(random_fract()) / n);
  }
  
! static double
  get_next_S(double t, int n, double *stateptr)
  {
  	double		S;
--- 1309,1322 ----
   * determines the number of records to skip before the next record is
   * processed.
   */
! double
  init_selection_state(int n)
  {
  	/* Initial value of W (for use when Algorithm Z is first applied) */
  	return exp(-log(random_fract()) / n);
  }
  
! double
  get_next_S(double t, int n, double *stateptr)
  {
  	double		S;
*************** compare_rows(const void *a, const void *
*** 1395,1401 ****
   */
  static int
  acquire_inherited_sample_rows(Relation onerel, HeapTuple *rows, int targrows,
! 							  double *totalrows, double *totaldeadrows)
  {
  	List	   *tableOIDs;
  	Relation   *rels;
--- 1432,1439 ----
   */
  static int
  acquire_inherited_sample_rows(Relation onerel, HeapTuple *rows, int targrows,
! 							  double *totalrows, double *totaldeadrows,
! 							  BlockNumber *totalpages, int elevel)
  {
  	List	   *tableOIDs;
  	Relation   *rels;
*************** acquire_inherited_sample_rows(Relation o
*** 1458,1463 ****
--- 1496,1503 ----
  		totalblocks += relblocks[nrels];
  		nrels++;
  	}
+ 	if (totalpages)
+ 	  *totalpages = totalblocks;
  
  	/*
  	 * Now sample rows from each relation, proportionally to its fraction of
*************** acquire_inherited_sample_rows(Relation o
*** 1491,1497 ****
  												rows + numrows,
  												childtargrows,
  												&trows,
! 												&tdrows);
  
  				/* We may need to convert from child's rowtype to parent's */
  				if (childrows > 0 &&
--- 1531,1539 ----
  												rows + numrows,
  												childtargrows,
  												&trows,
! 												&tdrows,
! 												NULL,
! 												elevel);
  
  				/* We may need to convert from child's rowtype to parent's */
  				if (childrows > 0 &&
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 9853686..3031496 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
*************** static void ATPrepSetStatistics(Relation
*** 320,325 ****
--- 320,327 ----
  					Node *newValue, LOCKMODE lockmode);
  static void ATExecSetStatistics(Relation rel, const char *colName,
  					Node *newValue, LOCKMODE lockmode);
+ static void ATPrepSetOptions(Relation rel, const char *colName,
+ 				 Node *options, LOCKMODE lockmode);
  static void ATExecSetOptions(Relation rel, const char *colName,
  				 Node *options, bool isReset, LOCKMODE lockmode);
  static void ATExecSetStorage(Relation rel, const char *colName,
*************** ATPrepCmd(List **wqueue, Relation rel, A
*** 3021,3027 ****
  			break;
  		case AT_SetOptions:		/* ALTER COLUMN SET ( options ) */
  		case AT_ResetOptions:	/* ALTER COLUMN RESET ( options ) */
! 			ATSimplePermissions(rel, ATT_TABLE | ATT_INDEX);
  			/* This command never recurses */
  			pass = AT_PASS_MISC;
  			break;
--- 3023,3030 ----
  			break;
  		case AT_SetOptions:		/* ALTER COLUMN SET ( options ) */
  		case AT_ResetOptions:	/* ALTER COLUMN RESET ( options ) */
! 			ATSimplePermissions(rel, ATT_TABLE | ATT_INDEX | ATT_FOREIGN_TABLE);
! 			ATPrepSetOptions(rel, cmd->name, cmd->def, lockmode);
  			/* This command never recurses */
  			pass = AT_PASS_MISC;
  			break;
*************** ATPrepSetStatistics(Relation rel, const 
*** 4999,5008 ****
  	 * allowSystemTableMods to be turned on.
  	 */
  	if (rel->rd_rel->relkind != RELKIND_RELATION &&
! 		rel->rd_rel->relkind != RELKIND_INDEX)
  		ereport(ERROR,
  				(errcode(ERRCODE_WRONG_OBJECT_TYPE),
! 				 errmsg("\"%s\" is not a table or index",
  						RelationGetRelationName(rel))));
  
  	/* Permissions checks */
--- 5002,5012 ----
  	 * allowSystemTableMods to be turned on.
  	 */
  	if (rel->rd_rel->relkind != RELKIND_RELATION &&
! 		rel->rd_rel->relkind != RELKIND_INDEX &&
! 		rel->rd_rel->relkind != RELKIND_FOREIGN_TABLE)
  		ereport(ERROR,
  				(errcode(ERRCODE_WRONG_OBJECT_TYPE),
! 				 errmsg("\"%s\" is not a table, index or foreign table",
  						RelationGetRelationName(rel))));
  
  	/* Permissions checks */
*************** ATExecSetStatistics(Relation rel, const 
*** 5071,5076 ****
--- 5075,5100 ----
  }
  
  static void
+ ATPrepSetOptions(Relation rel, const char *colName, Node *options,
+ 				 LOCKMODE lockmode)
+ {
+ 	if (rel->rd_rel->relkind == RELKIND_FOREIGN_TABLE)
+ 	{
+ 		ListCell   *cell;
+ 
+ 		foreach(cell, (List *) options)
+ 		{
+ 			DefElem    *def = (DefElem *) lfirst(cell);
+ 
+ 			if (pg_strcasecmp(def->defname, "n_distinct_inherited") == 0)
+ 				ereport(ERROR,
+ 						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ 						 errmsg("cannot set \"n_distinct_inherited\" for foreign tables")));
+ 		}
+ 	}
+ }
+ 
+ static void
  ATExecSetOptions(Relation rel, const char *colName, Node *options,
  				 bool isReset, LOCKMODE lockmode)
  {
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index dc2248b..b3d2078 100644
*** a/src/bin/psql/describe.c
--- b/src/bin/psql/describe.c
*************** describeOneTableDetails(const char *sche
*** 1104,1110 ****
  	bool		printTableInitialized = false;
  	int			i;
  	char	   *view_def = NULL;
! 	char	   *headers[6];
  	char	  **seq_values = NULL;
  	char	  **modifiers = NULL;
  	char	  **ptr;
--- 1104,1110 ----
  	bool		printTableInitialized = false;
  	int			i;
  	char	   *view_def = NULL;
! 	char	   *headers[7];
  	char	  **seq_values = NULL;
  	char	  **modifiers = NULL;
  	char	  **ptr;
*************** describeOneTableDetails(const char *sche
*** 1395,1401 ****
  	if (verbose)
  	{
  		headers[cols++] = gettext_noop("Storage");
! 		if (tableinfo.relkind == 'r')
  			headers[cols++] = gettext_noop("Stats target");
  		/* Column comments, if the relkind supports this feature. */
  		if (tableinfo.relkind == 'r' || tableinfo.relkind == 'v' ||
--- 1395,1401 ----
  	if (verbose)
  	{
  		headers[cols++] = gettext_noop("Storage");
! 		if (tableinfo.relkind == 'r' || tableinfo.relkind == 'f')
  			headers[cols++] = gettext_noop("Stats target");
  		/* Column comments, if the relkind supports this feature. */
  		if (tableinfo.relkind == 'r' || tableinfo.relkind == 'v' ||
*************** describeOneTableDetails(const char *sche
*** 1498,1504 ****
  							  false, false);
  
  			/* Statistics target, if the relkind supports this feature */
! 			if (tableinfo.relkind == 'r')
  			{
  				printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
  								  false, false);
--- 1498,1504 ----
  							  false, false);
  
  			/* Statistics target, if the relkind supports this feature */
! 			if (tableinfo.relkind == 'r' || tableinfo.relkind == 'f')
  			{
  				printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
  								  false, false);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 975d655..d113adf 100644
*** a/src/bin/psql/tab-complete.c
--- b/src/bin/psql/tab-complete.c
*************** static const SchemaQuery Query_for_list_
*** 409,414 ****
--- 409,429 ----
  	NULL
  };
  
+ static const SchemaQuery Query_for_list_of_tf = {
+ 	/* catname */
+ 	"pg_catalog.pg_class c",
+ 	/* selcondition */
+ 	"c.relkind IN ('r', 'f')",
+ 	/* viscondition */
+ 	"pg_catalog.pg_table_is_visible(c.oid)",
+ 	/* namespace */
+ 	"c.relnamespace",
+ 	/* result */
+ 	"pg_catalog.quote_ident(c.relname)",
+ 	/* qualresult */
+ 	NULL
+ };
+ 
  static const SchemaQuery Query_for_list_of_views = {
  	/* catname */
  	"pg_catalog.pg_class c",
*************** psql_completion(char *text, int start, i
*** 2833,2839 ****
  /* ANALYZE */
  	/* If the previous word is ANALYZE, produce list of tables */
  	else if (pg_strcasecmp(prev_wd, "ANALYZE") == 0)
! 		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_tables, NULL);
  
  /* WHERE */
  	/* Simple case of the word before the where being the table name */
--- 2848,2854 ----
  /* ANALYZE */
  	/* If the previous word is ANALYZE, produce list of tables */
  	else if (pg_strcasecmp(prev_wd, "ANALYZE") == 0)
! 		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_tf, NULL);
  
  /* WHERE */
  	/* Simple case of the word before the where being the table name */
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 3deee66..3e89e82 100644
*** a/src/include/commands/vacuum.h
--- b/src/include/commands/vacuum.h
*************** extern void lazy_vacuum_rel(Relation one
*** 170,174 ****
--- 170,183 ----
  extern void analyze_rel(Oid relid, VacuumStmt *vacstmt,
  			BufferAccessStrategy bstrategy);
  extern bool std_typanalyze(VacAttrStats *stats);
+ typedef int (*SampleRowAcquireFunc) (Relation onerel, HeapTuple *rows,
+ 									 int targrows, double *totalrows,
+ 									 double *totaldeadrows,
+ 									 BlockNumber *totalpages, int elevel);
+ extern void do_analyze_rel(Relation onerel, VacuumStmt *vacstmt, int elevel,
+ 						   bool inh, SampleRowAcquireFunc acquirefunc);
+ extern double random_fract(void);
+ extern double init_selection_state(int n);
+ extern double get_next_S(double t, int n, double *stateptr);
  
  #endif   /* VACUUM_H */
diff --git a/src/include/foreign/fdwapi.h b/src/include/foreign/fdwapi.h
index 854f177..d7181c7 100644
*** a/src/include/foreign/fdwapi.h
--- b/src/include/foreign/fdwapi.h
***************
*** 12,19 ****
--- 12,21 ----
  #ifndef FDWAPI_H
  #define FDWAPI_H
  
+ #include "foreign/foreign.h"
  #include "nodes/execnodes.h"
  #include "nodes/relation.h"
+ #include "utils/rel.h"
  
  /* To avoid including explain.h here, reference ExplainState thus: */
  struct ExplainState;
*************** typedef void (*ReScanForeignScan_functio
*** 50,55 ****
--- 52,60 ----
  
  typedef void (*EndForeignScan_function) (ForeignScanState *node);
  
+ typedef void (*AnalyzeForeignTable_function) (Relation relation,
+ 											  VacuumStmt *vacstmt,
+ 											  int elevel);
  
  /*
   * FdwRoutine is the struct returned by a foreign-data wrapper's handler
*************** typedef struct FdwRoutine
*** 64,69 ****
--- 69,78 ----
  {
  	NodeTag		type;
  
+ 	/*
+ 	 * These handlers are required to execute a scan on a foreign table.  If
+ 	 * any of them was NULL, scans on a foreign table managed by such FDW fail.
+ 	 */
  	GetForeignRelSize_function GetForeignRelSize;
  	GetForeignPaths_function GetForeignPaths;
  	GetForeignPlan_function GetForeignPlan;
*************** typedef struct FdwRoutine
*** 72,77 ****
--- 81,92 ----
  	IterateForeignScan_function IterateForeignScan;
  	ReScanForeignScan_function ReScanForeignScan;
  	EndForeignScan_function EndForeignScan;
+ 
+ 	/*
+ 	 * Handlers below are optional.  You can set any of them to NULL to tell
+ 	 * PostgreSQL backend that the FDW doesn't have the capability.
+ 	 */
+ 	AnalyzeForeignTable_function AnalyzeForeignTable;
  } FdwRoutine;
  
  
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index ba86883..f1379a6 100644
*** a/src/test/regress/expected/foreign_data.out
--- b/src/test/regress/expected/foreign_data.out
*************** CREATE FOREIGN TABLE ft1 (
*** 679,690 ****
  COMMENT ON FOREIGN TABLE ft1 IS 'ft1';
  COMMENT ON COLUMN ft1.c1 IS 'ft1.c1';
  \d+ ft1
!                                Foreign table "public.ft1"
!  Column |  Type   | Modifiers |          FDW Options           | Storage  | Description 
! --------+---------+-----------+--------------------------------+----------+-------------
!  c1     | integer | not null  | ("param 1" 'val1')             | plain    | ft1.c1
!  c2     | text    |           | (param2 'val2', param3 'val3') | extended | 
!  c3     | date    |           |                                | plain    | 
  Server: s0
  FDW Options: (delimiter ',', quote '"', "be quoted" 'value')
  Has OIDs: no
--- 679,690 ----
  COMMENT ON FOREIGN TABLE ft1 IS 'ft1';
  COMMENT ON COLUMN ft1.c1 IS 'ft1.c1';
  \d+ ft1
!                                       Foreign table "public.ft1"
!  Column |  Type   | Modifiers |          FDW Options           | Storage  | Stats target | Description 
! --------+---------+-----------+--------------------------------+----------+--------------+-------------
!  c1     | integer | not null  | ("param 1" 'val1')             | plain    |              | ft1.c1
!  c2     | text    |           | (param2 'val2', param3 'val3') | extended |              | 
!  c3     | date    |           |                                | plain    |              | 
  Server: s0
  FDW Options: (delimiter ',', quote '"', "be quoted" 'value')
  Has OIDs: no
*************** ERROR:  cannot alter system column "xmin
*** 730,748 ****
  ALTER FOREIGN TABLE ft1 ALTER COLUMN c7 OPTIONS (ADD p1 'v1', ADD p2 'v2'),
                          ALTER COLUMN c8 OPTIONS (ADD p1 'v1', ADD p2 'v2');
  ALTER FOREIGN TABLE ft1 ALTER COLUMN c8 OPTIONS (SET p2 'V2', DROP p1);
  \d+ ft1
!                                Foreign table "public.ft1"
!  Column |  Type   | Modifiers |          FDW Options           | Storage  | Description 
! --------+---------+-----------+--------------------------------+----------+-------------
!  c1     | integer | not null  | ("param 1" 'val1')             | plain    | 
!  c2     | text    |           | (param2 'val2', param3 'val3') | extended | 
!  c3     | date    |           |                                | plain    | 
!  c4     | integer |           |                                | plain    | 
!  c6     | integer | not null  |                                | plain    | 
!  c7     | integer |           | (p1 'v1', p2 'v2')             | plain    | 
!  c8     | text    |           | (p2 'V2')                      | extended | 
!  c9     | integer |           |                                | plain    | 
!  c10    | integer |           | (p1 'v1')                      | plain    | 
  Server: s0
  FDW Options: (delimiter ',', quote '"', "be quoted" 'value')
  Has OIDs: no
--- 730,753 ----
  ALTER FOREIGN TABLE ft1 ALTER COLUMN c7 OPTIONS (ADD p1 'v1', ADD p2 'v2'),
                          ALTER COLUMN c8 OPTIONS (ADD p1 'v1', ADD p2 'v2');
  ALTER FOREIGN TABLE ft1 ALTER COLUMN c8 OPTIONS (SET p2 'V2', DROP p1);
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c1 SET STATISTICS 10000;
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c1 SET (n_distinct = 100);
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c1 SET (n_distinct_inherited = 100); -- ERROR
+ ERROR:  cannot set "n_distinct_inherited" for foreign tables
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c8 SET STATISTICS -1;
  \d+ ft1
!                                       Foreign table "public.ft1"
!  Column |  Type   | Modifiers |          FDW Options           | Storage  | Stats target | Description 
! --------+---------+-----------+--------------------------------+----------+--------------+-------------
!  c1     | integer | not null  | ("param 1" 'val1')             | plain    | 10000        | 
!  c2     | text    |           | (param2 'val2', param3 'val3') | extended |              | 
!  c3     | date    |           |                                | plain    |              | 
!  c4     | integer |           |                                | plain    |              | 
!  c6     | integer | not null  |                                | plain    |              | 
!  c7     | integer |           | (p1 'v1', p2 'v2')             | plain    |              | 
!  c8     | text    |           | (p2 'V2')                      | extended |              | 
!  c9     | integer |           |                                | plain    |              | 
!  c10    | integer |           | (p1 'v1')                      | plain    |              | 
  Server: s0
  FDW Options: (delimiter ',', quote '"', "be quoted" 'value')
  Has OIDs: no
diff --git a/src/test/regress/sql/foreign_data.sql b/src/test/regress/sql/foreign_data.sql
index 0c95672..03b5680 100644
*** a/src/test/regress/sql/foreign_data.sql
--- b/src/test/regress/sql/foreign_data.sql
*************** ALTER FOREIGN TABLE ft1 ALTER COLUMN xmi
*** 307,312 ****
--- 307,316 ----
  ALTER FOREIGN TABLE ft1 ALTER COLUMN c7 OPTIONS (ADD p1 'v1', ADD p2 'v2'),
                          ALTER COLUMN c8 OPTIONS (ADD p1 'v1', ADD p2 'v2');
  ALTER FOREIGN TABLE ft1 ALTER COLUMN c8 OPTIONS (SET p2 'V2', DROP p1);
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c1 SET STATISTICS 10000;
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c1 SET (n_distinct = 100);
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c1 SET (n_distinct_inherited = 100); -- ERROR
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c8 SET STATISTICS -1;
  \d+ ft1
  -- can't change the column type if it's used elsewhere
  CREATE TABLE use_ft1_column_type (x ft1);


Attachments:

  [text/plain] postgresql-analyze-v7.patch (47.5K, ../4F7D8BBF.4050004@gmail.com/2-postgresql-analyze-v7.patch)
  download | inline diff:
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index e890770..f84a01f 100644
*** a/contrib/file_fdw/file_fdw.c
--- b/contrib/file_fdw/file_fdw.c
***************
*** 20,25 ****
--- 20,26 ----
  #include "commands/copy.h"
  #include "commands/defrem.h"
  #include "commands/explain.h"
+ #include "commands/vacuum.h"
  #include "foreign/fdwapi.h"
  #include "foreign/foreign.h"
  #include "miscadmin.h"
*************** static void fileBeginForeignScan(Foreign
*** 123,128 ****
--- 124,132 ----
  static TupleTableSlot *fileIterateForeignScan(ForeignScanState *node);
  static void fileReScanForeignScan(ForeignScanState *node);
  static void fileEndForeignScan(ForeignScanState *node);
+ static void fileAnalyzeForeignTable(Relation onerel,
+ 					VacuumStmt *vacstmt,
+ 					int elevel);
  
  /*
   * Helper functions
*************** static void estimate_size(PlannerInfo *r
*** 136,142 ****
  static void estimate_costs(PlannerInfo *root, RelOptInfo *baserel,
  			   FileFdwPlanState *fdw_private,
  			   Cost *startup_cost, Cost *total_cost);
! 
  
  /*
   * Foreign-data wrapper handler function: return a struct with pointers
--- 140,149 ----
  static void estimate_costs(PlannerInfo *root, RelOptInfo *baserel,
  			   FileFdwPlanState *fdw_private,
  			   Cost *startup_cost, Cost *total_cost);
! static int	acquire_sample_rows(Relation onerel,
! 			   HeapTuple *rows, int targrows,
! 			   double *totalrows, double *totaldeadrows,
! 			   BlockNumber *totalpages, int elevel);
  
  /*
   * Foreign-data wrapper handler function: return a struct with pointers
*************** file_fdw_handler(PG_FUNCTION_ARGS)
*** 155,160 ****
--- 162,168 ----
  	fdwroutine->IterateForeignScan = fileIterateForeignScan;
  	fdwroutine->ReScanForeignScan = fileReScanForeignScan;
  	fdwroutine->EndForeignScan = fileEndForeignScan;
+ 	fdwroutine->AnalyzeForeignTable = fileAnalyzeForeignTable;
  
  	PG_RETURN_POINTER(fdwroutine);
  }
*************** estimate_size(PlannerInfo *root, RelOptI
*** 662,693 ****
  	double		nrows;
  
  	/*
! 	 * Get size of the file.  It might not be there at plan time, though, in
! 	 * which case we have to use a default estimate.
  	 */
! 	if (stat(fdw_private->filename, &stat_buf) < 0)
! 		stat_buf.st_size = 10 * BLCKSZ;
  
! 	/*
! 	 * Convert size to pages for use in I/O cost estimate later.
! 	 */
! 	pages = (stat_buf.st_size + (BLCKSZ - 1)) / BLCKSZ;
! 	if (pages < 1)
! 		pages = 1;
  
! 	fdw_private->pages = pages;
  
! 	/*
! 	 * Estimate the number of tuples in the file.  We back into this estimate
! 	 * using the planner's idea of the relation width; which is bogus if not
! 	 * all columns are being read, not to mention that the text representation
! 	 * of a row probably isn't the same size as its internal representation.
! 	 * FIXME later.
! 	 */
! 	tuple_width = MAXALIGN(baserel->width) + MAXALIGN(sizeof(HeapTupleHeaderData));
  
! 	ntuples = clamp_row_est((double) stat_buf.st_size / (double) tuple_width);
  
  	fdw_private->ntuples = ntuples;
  
  	/*
--- 670,716 ----
  	double		nrows;
  
  	/*
! 	 * Use statistics stored in pg_class as is if any.  Otherwise, calculate
! 	 * them from file size and average tuple width.
  	 */
! 	if (baserel->pages > 0)
! 	{
! 		pages = baserel->pages;
! 		ntuples = baserel->tuples;
! 	}
! 	else
! 	{
! 		/*
! 		 * Get size of the file.  It might not be there at plan time, though,
! 		 * in which case we have to use a default estimate.
! 		 */
! 		if (stat(fdw_private->filename, &stat_buf) < 0)
! 			stat_buf.st_size = 10 * BLCKSZ;
  
! 		/*
! 		 * Convert size to pages for use in I/O cost estimate later.
! 		 */
! 		pages = (stat_buf.st_size + (BLCKSZ - 1)) / BLCKSZ;
! 		if (pages < 1)
! 			pages = 1;
  
! 		/*
! 		 * Estimate the number of tuples in the file.  We back into this
! 		 * estimate using the planner's idea of the relation width; which is
! 		 * bogus if not all columns are being read, not to mention that the
! 		 * text representation of a row probably isn't the same size as its
! 		 * internal representation.  FIXME later.
! 		 */
! 		tuple_width = MAXALIGN(baserel->width) +
! 					  MAXALIGN(sizeof(HeapTupleHeaderData));
  
! 		ntuples = clamp_row_est((double) stat_buf.st_size /
! 								(double) tuple_width);
  
! 	}
  
+ 	/* Pass estimates to subsequent functions via FileFdwPlanState. */
+ 	fdw_private->pages = pages;
  	fdw_private->ntuples = ntuples;
  
  	/*
*************** estimate_size(PlannerInfo *root, RelOptI
*** 709,714 ****
--- 732,747 ----
  }
  
  /*
+  * fileAnalyzeForeignTable
+  *		Analyze foreign table
+  */
+ static void
+ fileAnalyzeForeignTable(Relation onerel, VacuumStmt *vacstmt, int elevel)
+ {
+ 	do_analyze_rel(onerel, vacstmt, elevel, false, acquire_sample_rows);
+ }
+ 
+ /*
   * Estimate costs of scanning a foreign table.
   *
   * Results are returned in *startup_cost and *total_cost.
*************** estimate_costs(PlannerInfo *root, RelOpt
*** 736,738 ****
--- 769,957 ----
  	run_cost += cpu_per_tuple * ntuples;
  	*total_cost = *startup_cost + run_cost;
  }
+ 
+ /*
+  * acquire_sample_rows -- acquire a random sample of rows from the table
+  *
+  * Selected rows are returned in the caller-allocated array rows[], which must
+  * have at least targrows entries.  The actual number of rows selected is 
+  * returned as the function result.  We also count the number of valid rows in
+  * the table, and return it into *totalrows.
+  *
+  * The returned list of tuples is in order by physical position in the table.
+  * (We will rely on this later to derive correlation estimates.)
+  */
+ static int
+ acquire_sample_rows(Relation onerel, HeapTuple *rows, int targrows,
+ 					double *totalrows, double *totaldeadrows,
+ 					BlockNumber *totalpages, int elevel)
+ {
+ 	int			numrows = 0;
+ 	int			invalrows = 0; /* total # rows violating
+ 								  the NOT NULL constraints */
+ 	double		validrows = 0; /* total # rows collected */
+ 	double		rowstoskip = -1; /* -1 means not set yet */
+ 	double		rstate;
+ 	HeapTuple	tuple;
+ 	TupleDesc	tupDesc;
+ 	TupleConstr *constr;
+ 	int			natts;
+ 	int			attrChk;
+ 	Datum	   *values;
+ 	bool	   *nulls;
+ 	bool		found;
+ 	bool		sample_it = false;
+ 	char	   *filename;
+ 	struct stat	stat_buf;
+ 	List	   *options;
+ 	CopyState	cstate;
+ 	ErrorContextCallback errcontext;
+ 
+ 	Assert(onerel);
+ 	Assert(targrows > 0);
+ 
+ 	tupDesc = RelationGetDescr(onerel);
+ 	constr = tupDesc->constr;
+ 	natts = tupDesc->natts;
+ 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
+ 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
+ 
+ 	/* Fetch options of foreign table */
+ 	fileGetOptions(RelationGetRelid(onerel), &filename, &options);
+ 
+ 	/*
+ 	 * Get size of the file.
+ 	 */
+ 	if (stat(filename, &stat_buf) < 0)
+ 		ereport(ERROR,
+ 				(errcode_for_file_access(),
+ 				 errmsg("could not stat file \"%s\": %m",
+ 						filename)));
+ 
+ 	/*
+ 	 * Convert size to pages for use in I/O cost estimate.
+ 	 */
+ 	*totalpages = (stat_buf.st_size + (BLCKSZ - 1)) / BLCKSZ;
+ 	if (*totalpages < 1)
+ 		*totalpages = 1;
+ 
+ 	/*
+ 	 * Create CopyState from FDW options.  We always acquire all columns, so
+ 	 * as to match the expected ScanTupleSlot signature.
+ 	 */
+ 	cstate = BeginCopyFrom(onerel, filename, NIL, options);
+ 
+ 	/* Prepare for sampling rows */
+ 	rstate = init_selection_state(targrows);
+ 
+ 	/* Set up callback to identify error line number. */
+ 	errcontext.callback = CopyFromErrorCallback;
+ 	errcontext.arg = (void *) cstate;
+ 	errcontext.previous = error_context_stack;
+ 	error_context_stack = &errcontext;
+ 
+ 	for (;;)
+ 	{
+ 		sample_it = true;
+ 
+ 		/*
+ 		 * Check for user-requested abort.
+ 		 */
+ 		CHECK_FOR_INTERRUPTS();
+ 
+ 		found = NextCopyFrom(cstate, NULL, values, nulls, NULL);
+ 
+ 		if (!found)
+ 			break;
+ 
+ 		tuple = heap_form_tuple(tupDesc, values, nulls);
+ 
+ 		if (constr && constr->has_not_null)
+ 		{
+ 			for (attrChk = 1; attrChk <= natts; attrChk++)
+ 			{
+ 				if (onerel->rd_att->attrs[attrChk - 1]->attnotnull &&
+ 					heap_attisnull(tuple, attrChk))
+ 				{
+ 					sample_it = false;
+ 					break;
+ 				}
+ 			}
+ 		}
+ 
+ 		if (!sample_it)
+ 		{
+ 			invalrows += 1;
+ 			heap_freetuple(tuple);
+ 			continue;
+ 		}
+ 
+ 		/*
+ 		 * The first targrows sample rows are simply copied into the
+ 		 * reservoir. Then we start replacing tuples in the sample
+ 		 * until we reach the end of the relation.	This algorithm is
+ 		 * from Jeff Vitter's paper (see full citation below). It
+ 		 * works by repeatedly computing the number of tuples to skip
+ 		 * before selecting a tuple, which replaces a randomly chosen
+ 		 * element of the reservoir (current set of tuples).  At all
+ 		 * times the reservoir is a true random sample of the tuples
+ 		 * we've passed over so far, so when we fall off the end of
+ 		 * the relation we're done.
+ 		 */
+ 		if (numrows < targrows)
+ 			rows[numrows++] = heap_copytuple(tuple);
+ 		else
+ 		{
+ 			/*
+ 			 * t in Vitter's paper is the number of records already
+ 			 * processed.  If we need to compute a new S value, we
+ 			 * must use the not-yet-incremented value of samplerows as
+ 			 * t.
+ 			 */
+ 			if (rowstoskip < 0)
+ 				rowstoskip = get_next_S(validrows, targrows, &rstate);
+ 
+ 			if (rowstoskip <= 0)
+ 			{
+ 				/*
+ 				 * Found a suitable tuple, so save it, replacing one
+ 				 * old tuple at random
+ 				 */
+ 				int k = (int) (targrows * random_fract());
+ 
+ 				Assert(k >= 0 && k < targrows);
+ 				heap_freetuple(rows[k]);
+ 				rows[k] = heap_copytuple(tuple);
+ 			}
+ 
+ 			rowstoskip -= 1;
+ 		}
+ 
+ 		validrows += 1;
+ 		heap_freetuple(tuple);
+ 	}
+ 
+ 	/* Remove error callback. */
+ 	error_context_stack = errcontext.previous;
+ 
+ 	*totalrows = validrows;
+ 	*totaldeadrows = 0;
+ 
+ 	EndCopyFrom(cstate);
+ 
+ 	pfree(values);
+ 	pfree(nulls);
+ 
+ 	/*
+ 	 * Emit some interesting relation info
+ 	 */
+ 	ereport(elevel,
+ 			(errmsg("\"%s\": scanned, "
+ 					"containing %d valid rows and %d invalid rows; "
+ 					"%d rows in sample, %d total rows",
+ 					RelationGetRelationName(onerel),
+ 					(int) validrows, invalrows,
+ 					numrows, (int) *totalrows)));
+ 
+ 	return numrows;
+ }
diff --git a/contrib/file_fdw/input/file_fdw.source b/contrib/file_fdw/input/file_fdw.source
index 8e3d553..fddd9cd 100644
*** a/contrib/file_fdw/input/file_fdw.source
--- b/contrib/file_fdw/input/file_fdw.source
*************** EXECUTE st(100);
*** 111,116 ****
--- 111,121 ----
  EXECUTE st(100);
  DEALLOCATE st;
  
+ -- statistics collection tests
+ ANALYZE agg_csv;
+ SELECT relpages, reltuples FROM pg_class WHERE relname = 'agg_csv';
+ SELECT * FROM pg_stats WHERE tablename = 'agg_csv' ORDER BY attname;
+ 
  -- tableoid
  SELECT tableoid::regclass, b FROM agg_csv;
  
diff --git a/contrib/file_fdw/output/file_fdw.source b/contrib/file_fdw/output/file_fdw.source
index 84f0750..5be5fe0 100644
*** a/contrib/file_fdw/output/file_fdw.source
--- b/contrib/file_fdw/output/file_fdw.source
*************** EXECUTE st(100);
*** 174,179 ****
--- 174,194 ----
  (1 row)
  
  DEALLOCATE st;
+ -- statistics collection tests
+ ANALYZE agg_csv;
+ SELECT relpages, reltuples FROM pg_class WHERE relname = 'agg_csv';
+  relpages | reltuples 
+ ----------+-----------
+         1 |         3
+ (1 row)
+ 
+ SELECT * FROM pg_stats WHERE tablename = 'agg_csv' ORDER BY attname;
+  schemaname | tablename | attname | inherited | null_frac | avg_width | n_distinct | most_common_vals | most_common_freqs |    histogram_bounds     | correlation | most_common_elems | most_common_elem_freqs | elem_count_histogram 
+ ------------+-----------+---------+-----------+-----------+-----------+------------+------------------+-------------------+-------------------------+-------------+-------------------+------------------------+----------------------
+  public     | agg_csv   | a       | f         |         0 |         2 |         -1 |                  |                   | {0,42,100}              |        -0.5 |                   |                        | 
+  public     | agg_csv   | b       | f         |         0 |         4 |         -1 |                  |                   | {0.09561,99.097,324.78} |         0.5 |                   |                        | 
+ (2 rows)
+ 
  -- tableoid
  SELECT tableoid::regclass, b FROM agg_csv;
   tableoid |    b    
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index f7bf3d8..4f962b7 100644
*** a/doc/src/sgml/fdwhandler.sgml
--- b/doc/src/sgml/fdwhandler.sgml
*************** EndForeignScan (ForeignScanState *node);
*** 277,282 ****
--- 277,305 ----
      </para>
  
      <para>
+ <programlisting>
+ void
+ AnalyzeForeignTable (Relation onerel,
+                      VacuumStmt *vacstmt,
+                      int elevel);
+ </programlisting>
+ 
+      Collect statistics on a foreign table and store the results in the
+      pg_class and pg_statistics system catalogs.
+      This is optional, and if implemented, called when <command>ANALYZE</>
+      command is run.  The statistics are used by the query planner in order to
+      make good choices of query plans.
+     </para>
+ 
+     <para>
+      The function can be implemented by writing a sampling function that
+      acquires a random sample of rows from an external data source and
+      then by calling <function>do_analyze_rel</>, where you should pass
+      the sampling function as an argument.
+      The function must be set to NULL if it isn't implemented.
+     </para>
+ 
+     <para>
       The <structname>FdwRoutine</> struct type is declared in
       <filename>src/include/foreign/fdwapi.h</>, which see for additional
       details.
diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 93c3ff5..54d0838 100644
*** a/doc/src/sgml/maintenance.sgml
--- b/doc/src/sgml/maintenance.sgml
***************
*** 284,289 ****
--- 284,293 ----
      <command>ANALYZE</> strictly as a function of the number of rows
      inserted or updated; it has no knowledge of whether that will lead
      to meaningful statistical changes.
+     Note that the autovacuum daemon does not issue <command>ANALYZE</>
+     commands on foreign tables.  It is recommended to run manually-managed
+     <command>ANALYZE</> commands as needed, which typically are executed
+     according to a schedule by cron or Task Scheduler scripts.
     </para>
  
     <para>
diff --git a/doc/src/sgml/ref/alter_foreign_table.sgml b/doc/src/sgml/ref/alter_foreign_table.sgml
index c4cdaa8..af5c0a8 100644
*** a/doc/src/sgml/ref/alter_foreign_table.sgml
--- b/doc/src/sgml/ref/alter_foreign_table.sgml
*************** ALTER FOREIGN TABLE [ IF EXISTS ] <repla
*** 36,41 ****
--- 36,44 ----
      DROP [ COLUMN ] [ IF EXISTS ] <replaceable class="PARAMETER">column</replaceable> [ RESTRICT | CASCADE ]
      ALTER [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> [ SET DATA ] TYPE <replaceable class="PARAMETER">type</replaceable>
      ALTER [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> { SET | DROP } NOT NULL
+     ALTER [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> SET STATISTICS <replaceable class="PARAMETER">integer</replaceable>
+     ALTER [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> SET ( <replaceable class="PARAMETER">attribute_option</replaceable> = <replaceable class="PARAMETER">value</replaceable> [, ... ] )
+     ALTER [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> RESET ( <replaceable class="PARAMETER">attribute_option</replaceable> [, ... ] )
      ALTER [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> OPTIONS ( [ ADD | SET | DROP ] <replaceable class="PARAMETER">option</replaceable> ['<replaceable class="PARAMETER">value</replaceable>'] [, ... ])
      OWNER TO <replaceable class="PARAMETER">new_owner</replaceable>
      OPTIONS ( [ ADD | SET | DROP ] <replaceable class="PARAMETER">option</replaceable> ['<replaceable class="PARAMETER">value</replaceable>'] [, ... ])
*************** ALTER FOREIGN TABLE [ IF EXISTS ] <repla
*** 104,109 ****
--- 107,156 ----
     </varlistentry>
  
     <varlistentry>
+     <term><literal>SET STATISTICS</literal></term>
+     <listitem>
+      <para>
+       This form
+       sets the per-column statistics-gathering target for subsequent
+       <xref linkend="sql-analyze"> operations.
+       The target can be set in the range 0 to 10000; alternatively, set it
+       to -1 to revert to using the system default statistics
+       target (<xref linkend="guc-default-statistics-target">).
+      </para>
+     </listitem>
+    </varlistentry>
+ 
+    <varlistentry>
+     <term><literal>SET ( <replaceable class="PARAMETER">attribute_option</replaceable> = <replaceable class="PARAMETER">value</replaceable> [, ... ] )</literal></term>
+     <term><literal>RESET ( <replaceable class="PARAMETER">attribute_option</replaceable> [, ... ] )</literal></term>
+     <listitem>
+      <para>
+       This form
+       sets or resets a per-attribute option.  Currently, the only defined
+       per-attribute option is <literal>n_distinct</>, which overrides
+       the number-of-distinct-values estimates made by subsequent
+       <xref linkend="sql-analyze"> operations. 
+       When set to a positive value, <command>ANALYZE</> will assume that
+       the column contains exactly the specified number of distinct nonnull
+       values.
+       When set to a negative value, which must be greater than or equal
+       to -1, <command>ANALYZE</> will assume that the number of distinct
+       nonnull values in the column is linear in the size of the foreign
+       table; the exact count is to be computed by multiplying the estimated
+       foreign table size by the absolute value of the given number.
+       For example,
+       a value of -1 implies that all values in the column are distinct,
+       while a value of -0.5 implies that each value appears twice on the
+       average.
+       This can be useful when the size of the foreign table changes over
+       time, since the multiplication by the number of rows in the foreign
+       table is not performed until query planning time.  Specify a value
+       of 0 to revert to estimating the number of distinct values normally.
+      </para>
+     </listitem>
+    </varlistentry>
+ 
+    <varlistentry>
      <term><literal>OWNER</literal></term>
      <listitem>
       <para>
diff --git a/doc/src/sgml/ref/analyze.sgml b/doc/src/sgml/ref/analyze.sgml
index 8c9057b..97e9ad6 100644
*** a/doc/src/sgml/ref/analyze.sgml
--- b/doc/src/sgml/ref/analyze.sgml
*************** ANALYZE [ VERBOSE ] [ <replaceable class
*** 39,47 ****
  
    <para>
     With no parameter, <command>ANALYZE</command> examines every table in the
!    current database.  With a parameter, <command>ANALYZE</command> examines
!    only that table.  It is further possible to give a list of column names,
!    in which case only the statistics for those columns are collected.
    </para>
   </refsect1>
  
--- 39,49 ----
  
    <para>
     With no parameter, <command>ANALYZE</command> examines every table in the
!    current database except for foreign tables.  With a parameter,
!    <command>ANALYZE</command> examines only that table.  For a foreign table,
!    it is necessary to specify the name of that table.  It is further possible
!    to give a list of column names, in which case only the statistics for those
!    columns are collected.
    </para>
   </refsect1>
  
*************** ANALYZE [ VERBOSE ] [ <replaceable class
*** 63,69 ****
      <listitem>
       <para>
        The name (possibly schema-qualified) of a specific table to
!       analyze. Defaults to all tables in the current database.
       </para>
      </listitem>
     </varlistentry>
--- 65,72 ----
      <listitem>
       <para>
        The name (possibly schema-qualified) of a specific table to
!       analyze. Defaults to all tables in the current database except
!       for foreign tables.
       </para>
      </listitem>
     </varlistentry>
*************** ANALYZE [ VERBOSE ] [ <replaceable class
*** 137,143 ****
     In rare situations, this non-determinism will cause the planner's
     choices of query plans to change after <command>ANALYZE</command> is run.
     To avoid this, raise the amount of statistics collected by
!    <command>ANALYZE</command>, as described below.
    </para>
  
    <para>
--- 140,148 ----
     In rare situations, this non-determinism will cause the planner's
     choices of query plans to change after <command>ANALYZE</command> is run.
     To avoid this, raise the amount of statistics collected by
!    <command>ANALYZE</command>, as described below.  Note that the time
!    needed to analyze on foreign tables depends on the implementation of
!    the foreign data wrapper via which such tables are attached.
    </para>
  
    <para>
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index 9cd6e67..6ac9aee 100644
*** a/src/backend/commands/analyze.c
--- b/src/backend/commands/analyze.c
***************
*** 23,28 ****
--- 23,29 ----
  #include "access/xact.h"
  #include "catalog/index.h"
  #include "catalog/indexing.h"
+ #include "catalog/pg_class.h"
  #include "catalog/pg_collation.h"
  #include "catalog/pg_inherits_fn.h"
  #include "catalog/pg_namespace.h"
***************
*** 30,35 ****
--- 31,38 ----
  #include "commands/tablecmds.h"
  #include "commands/vacuum.h"
  #include "executor/executor.h"
+ #include "foreign/foreign.h"
+ #include "foreign/fdwapi.h"
  #include "miscadmin.h"
  #include "nodes/nodeFuncs.h"
  #include "parser/parse_oper.h"
*************** typedef struct AnlIndexData
*** 78,91 ****
  int			default_statistics_target = 100;
  
  /* A few variables that don't seem worth passing around as parameters */
- static int	elevel = -1;
- 
  static MemoryContext anl_context = NULL;
  
  static BufferAccessStrategy vac_strategy;
  
  
- static void do_analyze_rel(Relation onerel, VacuumStmt *vacstmt, bool inh);
  static void BlockSampler_Init(BlockSampler bs, BlockNumber nblocks,
  				  int samplesize);
  static bool BlockSampler_HasMore(BlockSampler bs);
--- 81,91 ----
*************** static void compute_index_stats(Relation
*** 97,110 ****
  static VacAttrStats *examine_attribute(Relation onerel, int attnum,
  				  Node *index_expr);
  static int acquire_sample_rows(Relation onerel, HeapTuple *rows,
! 					int targrows, double *totalrows, double *totaldeadrows);
! static double random_fract(void);
! static double init_selection_state(int n);
! static double get_next_S(double t, int n, double *stateptr);
  static int	compare_rows(const void *a, const void *b);
  static int acquire_inherited_sample_rows(Relation onerel,
  							  HeapTuple *rows, int targrows,
! 							  double *totalrows, double *totaldeadrows);
  static void update_attstats(Oid relid, bool inh,
  				int natts, VacAttrStats **vacattrstats);
  static Datum std_fetch_func(VacAttrStatsP stats, int rownum, bool *isNull);
--- 97,109 ----
  static VacAttrStats *examine_attribute(Relation onerel, int attnum,
  				  Node *index_expr);
  static int acquire_sample_rows(Relation onerel, HeapTuple *rows,
! 					int targrows, double *totalrows, double *totaldeadrows,
! 					BlockNumber *totalpages, int elevel);
  static int	compare_rows(const void *a, const void *b);
  static int acquire_inherited_sample_rows(Relation onerel,
  							  HeapTuple *rows, int targrows,
! 							  double *totalrows, double *totaldeadrows,
! 							  BlockNumber *totalpages, int elevel);
  static void update_attstats(Oid relid, bool inh,
  				int natts, VacAttrStats **vacattrstats);
  static Datum std_fetch_func(VacAttrStatsP stats, int rownum, bool *isNull);
*************** void
*** 118,123 ****
--- 117,124 ----
  analyze_rel(Oid relid, VacuumStmt *vacstmt, BufferAccessStrategy bstrategy)
  {
  	Relation	onerel;
+ 	int			elevel;
+ 	FdwRoutine *fdwroutine;
  
  	/* Set up static variables */
  	if (vacstmt->options & VACOPT_VERBOSE)
*************** analyze_rel(Oid relid, VacuumStmt *vacst
*** 182,191 ****
  	}
  
  	/*
! 	 * Check that it's a plain table; we used to do this in get_rel_oids() but
! 	 * seems safer to check after we've locked the relation.
  	 */
! 	if (onerel->rd_rel->relkind != RELKIND_RELATION)
  	{
  		/* No need for a WARNING if we already complained during VACUUM */
  		if (!(vacstmt->options & VACOPT_VACUUM))
--- 183,194 ----
  	}
  
  	/*
! 	 * Check that it's a plain table or foreign table; we used to do this
! 	 * in get_rel_oids() but seems safer to check after we've locked the
! 	 * relation.
  	 */
! 	if (onerel->rd_rel->relkind != RELKIND_RELATION &&
! 		onerel->rd_rel->relkind != RELKIND_FOREIGN_TABLE)
  	{
  		/* No need for a WARNING if we already complained during VACUUM */
  		if (!(vacstmt->options & VACOPT_VACUUM))
*************** analyze_rel(Oid relid, VacuumStmt *vacst
*** 209,215 ****
  	}
  
  	/*
! 	 * We can ANALYZE any table except pg_statistic. See update_attstats
  	 */
  	if (RelationGetRelid(onerel) == StatisticRelationId)
  	{
--- 212,220 ----
  	}
  
  	/*
! 	 * We can ANALYZE any table except pg_statistic.  See update_attstats.
! 	 * In addition, we can ANALYZE foreign tables if AnalyzeForeignTable
! 	 * callback routines of underlying foreign-data wrappers are implemented.
  	 */
  	if (RelationGetRelid(onerel) == StatisticRelationId)
  	{
*************** analyze_rel(Oid relid, VacuumStmt *vacst
*** 217,222 ****
--- 222,241 ----
  		return;
  	}
  
+ 	if (onerel->rd_rel->relkind == RELKIND_FOREIGN_TABLE)
+ 	{
+  		fdwroutine = GetFdwRoutineByRelId(RelationGetRelid(onerel));
+ 
+ 		if (fdwroutine->AnalyzeForeignTable == NULL)
+ 		{
+ 			ereport(WARNING,
+ 					(errmsg("skipping \"%s\" --- underlying foreign-data wrapper cannot analyze it",
+ 							RelationGetRelationName(onerel))));
+ 			relation_close(onerel, ShareUpdateExclusiveLock);
+ 			return;
+ 		}
+ 	}
+ 
  	/*
  	 * OK, let's do it.  First let other backends know I'm in ANALYZE.
  	 */
*************** analyze_rel(Oid relid, VacuumStmt *vacst
*** 224,239 ****
  	MyPgXact->vacuumFlags |= PROC_IN_ANALYZE;
  	LWLockRelease(ProcArrayLock);
  
! 	/*
! 	 * Do the normal non-recursive ANALYZE.
! 	 */
! 	do_analyze_rel(onerel, vacstmt, false);
  
! 	/*
! 	 * If there are child tables, do recursive ANALYZE.
! 	 */
! 	if (onerel->rd_rel->relhassubclass)
! 		do_analyze_rel(onerel, vacstmt, true);
  
  	/*
  	 * Close source relation now, but keep lock so that no one deletes it
--- 243,280 ----
  	MyPgXact->vacuumFlags |= PROC_IN_ANALYZE;
  	LWLockRelease(ProcArrayLock);
  
! 	if (onerel->rd_rel->relkind == RELKIND_FOREIGN_TABLE)
! 	{
! 		ereport(elevel,
! 				(errmsg("analyzing \"%s.%s\"",
! 						get_namespace_name(RelationGetNamespace(onerel)),
! 						RelationGetRelationName(onerel))));
! 		fdwroutine->AnalyzeForeignTable(onerel, vacstmt, elevel);
! 	}
! 	else
! 	{
! 		/*
! 		 * Do the normal non-recursive ANALYZE.
! 		 */
! 		ereport(elevel,
! 				(errmsg("analyzing \"%s.%s\"",
! 						get_namespace_name(RelationGetNamespace(onerel)),
! 						RelationGetRelationName(onerel))));
! 		do_analyze_rel(onerel, vacstmt, elevel, false, acquire_sample_rows);
  
! 		/*
! 		 * If there are child tables, do recursive ANALYZE.
! 		 */
! 		if (onerel->rd_rel->relhassubclass)
! 		{
! 			ereport(elevel,
! 					(errmsg("analyzing \"%s.%s\" inheritance tree",
! 							get_namespace_name(RelationGetNamespace(onerel)),
! 							RelationGetRelationName(onerel))));
! 			do_analyze_rel(onerel, vacstmt, elevel, true,
! 						   acquire_inherited_sample_rows);
! 		}
! 	}
  
  	/*
  	 * Close source relation now, but keep lock so that no one deletes it
*************** analyze_rel(Oid relid, VacuumStmt *vacst
*** 255,262 ****
  /*
   *	do_analyze_rel() -- analyze one relation, recursively or not
   */
! static void
! do_analyze_rel(Relation onerel, VacuumStmt *vacstmt, bool inh)
  {
  	int			attr_cnt,
  				tcnt,
--- 296,304 ----
  /*
   *	do_analyze_rel() -- analyze one relation, recursively or not
   */
! void
! do_analyze_rel(Relation onerel, VacuumStmt *vacstmt, int elevel, 
! 			   bool inh, SampleRowAcquireFunc acquirefunc)
  {
  	int			attr_cnt,
  				tcnt,
*************** do_analyze_rel(Relation onerel, VacuumSt
*** 271,276 ****
--- 313,319 ----
  				numrows;
  	double		totalrows,
  				totaldeadrows;
+ 	BlockNumber totalpages;
  	HeapTuple  *rows;
  	PGRUsage	ru0;
  	TimestampTz starttime = 0;
*************** do_analyze_rel(Relation onerel, VacuumSt
*** 279,295 ****
  	int			save_sec_context;
  	int			save_nestlevel;
  
- 	if (inh)
- 		ereport(elevel,
- 				(errmsg("analyzing \"%s.%s\" inheritance tree",
- 						get_namespace_name(RelationGetNamespace(onerel)),
- 						RelationGetRelationName(onerel))));
- 	else
- 		ereport(elevel,
- 				(errmsg("analyzing \"%s.%s\"",
- 						get_namespace_name(RelationGetNamespace(onerel)),
- 						RelationGetRelationName(onerel))));
- 
  	/*
  	 * Set up a working context so that we can easily free whatever junk gets
  	 * created.
--- 322,327 ----
*************** do_analyze_rel(Relation onerel, VacuumSt
*** 447,457 ****
  	 */
  	rows = (HeapTuple *) palloc(targrows * sizeof(HeapTuple));
  	if (inh)
! 		numrows = acquire_inherited_sample_rows(onerel, rows, targrows,
! 												&totalrows, &totaldeadrows);
  	else
! 		numrows = acquire_sample_rows(onerel, rows, targrows,
! 									  &totalrows, &totaldeadrows);
  
  	/*
  	 * Compute the statistics.	Temporary results during the calculations for
--- 479,491 ----
  	 */
  	rows = (HeapTuple *) palloc(targrows * sizeof(HeapTuple));
  	if (inh)
! 		numrows = acquirefunc(onerel, rows, targrows,
! 							  &totalrows, &totaldeadrows,
! 							  NULL, elevel);
  	else
! 		numrows = acquirefunc(onerel, rows, targrows,
! 							  &totalrows, &totaldeadrows,
! 							  &totalpages, elevel);
  
  	/*
  	 * Compute the statistics.	Temporary results during the calculations for
*************** do_analyze_rel(Relation onerel, VacuumSt
*** 532,538 ****
  	 */
  	if (!inh)
  		vac_update_relstats(onerel,
! 							RelationGetNumberOfBlocks(onerel),
  							totalrows,
  							visibilitymap_count(onerel),
  							hasindex,
--- 566,572 ----
  	 */
  	if (!inh)
  		vac_update_relstats(onerel,
! 							totalpages,
  							totalrows,
  							visibilitymap_count(onerel),
  							hasindex,
*************** BlockSampler_Next(BlockSampler bs)
*** 1015,1021 ****
   */
  static int
  acquire_sample_rows(Relation onerel, HeapTuple *rows, int targrows,
! 					double *totalrows, double *totaldeadrows)
  {
  	int			numrows = 0;	/* # rows now in reservoir */
  	double		samplerows = 0; /* total # rows collected */
--- 1049,1056 ----
   */
  static int
  acquire_sample_rows(Relation onerel, HeapTuple *rows, int targrows,
! 					double *totalrows, double *totaldeadrows,
! 					BlockNumber *totalpages, int elevel)
  {
  	int			numrows = 0;	/* # rows now in reservoir */
  	double		samplerows = 0; /* total # rows collected */
*************** acquire_sample_rows(Relation onerel, Hea
*** 1030,1035 ****
--- 1065,1072 ----
  	Assert(targrows > 0);
  
  	totalblocks = RelationGetNumberOfBlocks(onerel);
+ 	if (totalpages)
+ 		*totalpages = totalblocks;
  
  	/* Need a cutoff xmin for HeapTupleSatisfiesVacuum */
  	OldestXmin = GetOldestXmin(onerel->rd_rel->relisshared, true);
*************** acquire_sample_rows(Relation onerel, Hea
*** 1252,1258 ****
  }
  
  /* Select a random value R uniformly distributed in (0 - 1) */
! static double
  random_fract(void)
  {
  	return ((double) random() + 1) / ((double) MAX_RANDOM_VALUE + 2);
--- 1289,1295 ----
  }
  
  /* Select a random value R uniformly distributed in (0 - 1) */
! double
  random_fract(void)
  {
  	return ((double) random() + 1) / ((double) MAX_RANDOM_VALUE + 2);
*************** random_fract(void)
*** 1272,1285 ****
   * determines the number of records to skip before the next record is
   * processed.
   */
! static double
  init_selection_state(int n)
  {
  	/* Initial value of W (for use when Algorithm Z is first applied) */
  	return exp(-log(random_fract()) / n);
  }
  
! static double
  get_next_S(double t, int n, double *stateptr)
  {
  	double		S;
--- 1309,1322 ----
   * determines the number of records to skip before the next record is
   * processed.
   */
! double
  init_selection_state(int n)
  {
  	/* Initial value of W (for use when Algorithm Z is first applied) */
  	return exp(-log(random_fract()) / n);
  }
  
! double
  get_next_S(double t, int n, double *stateptr)
  {
  	double		S;
*************** compare_rows(const void *a, const void *
*** 1395,1401 ****
   */
  static int
  acquire_inherited_sample_rows(Relation onerel, HeapTuple *rows, int targrows,
! 							  double *totalrows, double *totaldeadrows)
  {
  	List	   *tableOIDs;
  	Relation   *rels;
--- 1432,1439 ----
   */
  static int
  acquire_inherited_sample_rows(Relation onerel, HeapTuple *rows, int targrows,
! 							  double *totalrows, double *totaldeadrows,
! 							  BlockNumber *totalpages, int elevel)
  {
  	List	   *tableOIDs;
  	Relation   *rels;
*************** acquire_inherited_sample_rows(Relation o
*** 1458,1463 ****
--- 1496,1503 ----
  		totalblocks += relblocks[nrels];
  		nrels++;
  	}
+ 	if (totalpages)
+ 	  *totalpages = totalblocks;
  
  	/*
  	 * Now sample rows from each relation, proportionally to its fraction of
*************** acquire_inherited_sample_rows(Relation o
*** 1491,1497 ****
  												rows + numrows,
  												childtargrows,
  												&trows,
! 												&tdrows);
  
  				/* We may need to convert from child's rowtype to parent's */
  				if (childrows > 0 &&
--- 1531,1539 ----
  												rows + numrows,
  												childtargrows,
  												&trows,
! 												&tdrows,
! 												NULL,
! 												elevel);
  
  				/* We may need to convert from child's rowtype to parent's */
  				if (childrows > 0 &&
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 9853686..3031496 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
*************** static void ATPrepSetStatistics(Relation
*** 320,325 ****
--- 320,327 ----
  					Node *newValue, LOCKMODE lockmode);
  static void ATExecSetStatistics(Relation rel, const char *colName,
  					Node *newValue, LOCKMODE lockmode);
+ static void ATPrepSetOptions(Relation rel, const char *colName,
+ 				 Node *options, LOCKMODE lockmode);
  static void ATExecSetOptions(Relation rel, const char *colName,
  				 Node *options, bool isReset, LOCKMODE lockmode);
  static void ATExecSetStorage(Relation rel, const char *colName,
*************** ATPrepCmd(List **wqueue, Relation rel, A
*** 3021,3027 ****
  			break;
  		case AT_SetOptions:		/* ALTER COLUMN SET ( options ) */
  		case AT_ResetOptions:	/* ALTER COLUMN RESET ( options ) */
! 			ATSimplePermissions(rel, ATT_TABLE | ATT_INDEX);
  			/* This command never recurses */
  			pass = AT_PASS_MISC;
  			break;
--- 3023,3030 ----
  			break;
  		case AT_SetOptions:		/* ALTER COLUMN SET ( options ) */
  		case AT_ResetOptions:	/* ALTER COLUMN RESET ( options ) */
! 			ATSimplePermissions(rel, ATT_TABLE | ATT_INDEX | ATT_FOREIGN_TABLE);
! 			ATPrepSetOptions(rel, cmd->name, cmd->def, lockmode);
  			/* This command never recurses */
  			pass = AT_PASS_MISC;
  			break;
*************** ATPrepSetStatistics(Relation rel, const 
*** 4999,5008 ****
  	 * allowSystemTableMods to be turned on.
  	 */
  	if (rel->rd_rel->relkind != RELKIND_RELATION &&
! 		rel->rd_rel->relkind != RELKIND_INDEX)
  		ereport(ERROR,
  				(errcode(ERRCODE_WRONG_OBJECT_TYPE),
! 				 errmsg("\"%s\" is not a table or index",
  						RelationGetRelationName(rel))));
  
  	/* Permissions checks */
--- 5002,5012 ----
  	 * allowSystemTableMods to be turned on.
  	 */
  	if (rel->rd_rel->relkind != RELKIND_RELATION &&
! 		rel->rd_rel->relkind != RELKIND_INDEX &&
! 		rel->rd_rel->relkind != RELKIND_FOREIGN_TABLE)
  		ereport(ERROR,
  				(errcode(ERRCODE_WRONG_OBJECT_TYPE),
! 				 errmsg("\"%s\" is not a table, index or foreign table",
  						RelationGetRelationName(rel))));
  
  	/* Permissions checks */
*************** ATExecSetStatistics(Relation rel, const 
*** 5071,5076 ****
--- 5075,5100 ----
  }
  
  static void
+ ATPrepSetOptions(Relation rel, const char *colName, Node *options,
+ 				 LOCKMODE lockmode)
+ {
+ 	if (rel->rd_rel->relkind == RELKIND_FOREIGN_TABLE)
+ 	{
+ 		ListCell   *cell;
+ 
+ 		foreach(cell, (List *) options)
+ 		{
+ 			DefElem    *def = (DefElem *) lfirst(cell);
+ 
+ 			if (pg_strcasecmp(def->defname, "n_distinct_inherited") == 0)
+ 				ereport(ERROR,
+ 						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ 						 errmsg("cannot set \"n_distinct_inherited\" for foreign tables")));
+ 		}
+ 	}
+ }
+ 
+ static void
  ATExecSetOptions(Relation rel, const char *colName, Node *options,
  				 bool isReset, LOCKMODE lockmode)
  {
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index dc2248b..b3d2078 100644
*** a/src/bin/psql/describe.c
--- b/src/bin/psql/describe.c
*************** describeOneTableDetails(const char *sche
*** 1104,1110 ****
  	bool		printTableInitialized = false;
  	int			i;
  	char	   *view_def = NULL;
! 	char	   *headers[6];
  	char	  **seq_values = NULL;
  	char	  **modifiers = NULL;
  	char	  **ptr;
--- 1104,1110 ----
  	bool		printTableInitialized = false;
  	int			i;
  	char	   *view_def = NULL;
! 	char	   *headers[7];
  	char	  **seq_values = NULL;
  	char	  **modifiers = NULL;
  	char	  **ptr;
*************** describeOneTableDetails(const char *sche
*** 1395,1401 ****
  	if (verbose)
  	{
  		headers[cols++] = gettext_noop("Storage");
! 		if (tableinfo.relkind == 'r')
  			headers[cols++] = gettext_noop("Stats target");
  		/* Column comments, if the relkind supports this feature. */
  		if (tableinfo.relkind == 'r' || tableinfo.relkind == 'v' ||
--- 1395,1401 ----
  	if (verbose)
  	{
  		headers[cols++] = gettext_noop("Storage");
! 		if (tableinfo.relkind == 'r' || tableinfo.relkind == 'f')
  			headers[cols++] = gettext_noop("Stats target");
  		/* Column comments, if the relkind supports this feature. */
  		if (tableinfo.relkind == 'r' || tableinfo.relkind == 'v' ||
*************** describeOneTableDetails(const char *sche
*** 1498,1504 ****
  							  false, false);
  
  			/* Statistics target, if the relkind supports this feature */
! 			if (tableinfo.relkind == 'r')
  			{
  				printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
  								  false, false);
--- 1498,1504 ----
  							  false, false);
  
  			/* Statistics target, if the relkind supports this feature */
! 			if (tableinfo.relkind == 'r' || tableinfo.relkind == 'f')
  			{
  				printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
  								  false, false);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 975d655..d113adf 100644
*** a/src/bin/psql/tab-complete.c
--- b/src/bin/psql/tab-complete.c
*************** static const SchemaQuery Query_for_list_
*** 409,414 ****
--- 409,429 ----
  	NULL
  };
  
+ static const SchemaQuery Query_for_list_of_tf = {
+ 	/* catname */
+ 	"pg_catalog.pg_class c",
+ 	/* selcondition */
+ 	"c.relkind IN ('r', 'f')",
+ 	/* viscondition */
+ 	"pg_catalog.pg_table_is_visible(c.oid)",
+ 	/* namespace */
+ 	"c.relnamespace",
+ 	/* result */
+ 	"pg_catalog.quote_ident(c.relname)",
+ 	/* qualresult */
+ 	NULL
+ };
+ 
  static const SchemaQuery Query_for_list_of_views = {
  	/* catname */
  	"pg_catalog.pg_class c",
*************** psql_completion(char *text, int start, i
*** 2833,2839 ****
  /* ANALYZE */
  	/* If the previous word is ANALYZE, produce list of tables */
  	else if (pg_strcasecmp(prev_wd, "ANALYZE") == 0)
! 		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_tables, NULL);
  
  /* WHERE */
  	/* Simple case of the word before the where being the table name */
--- 2848,2854 ----
  /* ANALYZE */
  	/* If the previous word is ANALYZE, produce list of tables */
  	else if (pg_strcasecmp(prev_wd, "ANALYZE") == 0)
! 		COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_tf, NULL);
  
  /* WHERE */
  	/* Simple case of the word before the where being the table name */
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 3deee66..3e89e82 100644
*** a/src/include/commands/vacuum.h
--- b/src/include/commands/vacuum.h
*************** extern void lazy_vacuum_rel(Relation one
*** 170,174 ****
--- 170,183 ----
  extern void analyze_rel(Oid relid, VacuumStmt *vacstmt,
  			BufferAccessStrategy bstrategy);
  extern bool std_typanalyze(VacAttrStats *stats);
+ typedef int (*SampleRowAcquireFunc) (Relation onerel, HeapTuple *rows,
+ 									 int targrows, double *totalrows,
+ 									 double *totaldeadrows,
+ 									 BlockNumber *totalpages, int elevel);
+ extern void do_analyze_rel(Relation onerel, VacuumStmt *vacstmt, int elevel,
+ 						   bool inh, SampleRowAcquireFunc acquirefunc);
+ extern double random_fract(void);
+ extern double init_selection_state(int n);
+ extern double get_next_S(double t, int n, double *stateptr);
  
  #endif   /* VACUUM_H */
diff --git a/src/include/foreign/fdwapi.h b/src/include/foreign/fdwapi.h
index 854f177..d7181c7 100644
*** a/src/include/foreign/fdwapi.h
--- b/src/include/foreign/fdwapi.h
***************
*** 12,19 ****
--- 12,21 ----
  #ifndef FDWAPI_H
  #define FDWAPI_H
  
+ #include "foreign/foreign.h"
  #include "nodes/execnodes.h"
  #include "nodes/relation.h"
+ #include "utils/rel.h"
  
  /* To avoid including explain.h here, reference ExplainState thus: */
  struct ExplainState;
*************** typedef void (*ReScanForeignScan_functio
*** 50,55 ****
--- 52,60 ----
  
  typedef void (*EndForeignScan_function) (ForeignScanState *node);
  
+ typedef void (*AnalyzeForeignTable_function) (Relation relation,
+ 											  VacuumStmt *vacstmt,
+ 											  int elevel);
  
  /*
   * FdwRoutine is the struct returned by a foreign-data wrapper's handler
*************** typedef struct FdwRoutine
*** 64,69 ****
--- 69,78 ----
  {
  	NodeTag		type;
  
+ 	/*
+ 	 * These handlers are required to execute a scan on a foreign table.  If
+ 	 * any of them was NULL, scans on a foreign table managed by such FDW fail.
+ 	 */
  	GetForeignRelSize_function GetForeignRelSize;
  	GetForeignPaths_function GetForeignPaths;
  	GetForeignPlan_function GetForeignPlan;
*************** typedef struct FdwRoutine
*** 72,77 ****
--- 81,92 ----
  	IterateForeignScan_function IterateForeignScan;
  	ReScanForeignScan_function ReScanForeignScan;
  	EndForeignScan_function EndForeignScan;
+ 
+ 	/*
+ 	 * Handlers below are optional.  You can set any of them to NULL to tell
+ 	 * PostgreSQL backend that the FDW doesn't have the capability.
+ 	 */
+ 	AnalyzeForeignTable_function AnalyzeForeignTable;
  } FdwRoutine;
  
  
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index ba86883..f1379a6 100644
*** a/src/test/regress/expected/foreign_data.out
--- b/src/test/regress/expected/foreign_data.out
*************** CREATE FOREIGN TABLE ft1 (
*** 679,690 ****
  COMMENT ON FOREIGN TABLE ft1 IS 'ft1';
  COMMENT ON COLUMN ft1.c1 IS 'ft1.c1';
  \d+ ft1
!                                Foreign table "public.ft1"
!  Column |  Type   | Modifiers |          FDW Options           | Storage  | Description 
! --------+---------+-----------+--------------------------------+----------+-------------
!  c1     | integer | not null  | ("param 1" 'val1')             | plain    | ft1.c1
!  c2     | text    |           | (param2 'val2', param3 'val3') | extended | 
!  c3     | date    |           |                                | plain    | 
  Server: s0
  FDW Options: (delimiter ',', quote '"', "be quoted" 'value')
  Has OIDs: no
--- 679,690 ----
  COMMENT ON FOREIGN TABLE ft1 IS 'ft1';
  COMMENT ON COLUMN ft1.c1 IS 'ft1.c1';
  \d+ ft1
!                                       Foreign table "public.ft1"
!  Column |  Type   | Modifiers |          FDW Options           | Storage  | Stats target | Description 
! --------+---------+-----------+--------------------------------+----------+--------------+-------------
!  c1     | integer | not null  | ("param 1" 'val1')             | plain    |              | ft1.c1
!  c2     | text    |           | (param2 'val2', param3 'val3') | extended |              | 
!  c3     | date    |           |                                | plain    |              | 
  Server: s0
  FDW Options: (delimiter ',', quote '"', "be quoted" 'value')
  Has OIDs: no
*************** ERROR:  cannot alter system column "xmin
*** 730,748 ****
  ALTER FOREIGN TABLE ft1 ALTER COLUMN c7 OPTIONS (ADD p1 'v1', ADD p2 'v2'),
                          ALTER COLUMN c8 OPTIONS (ADD p1 'v1', ADD p2 'v2');
  ALTER FOREIGN TABLE ft1 ALTER COLUMN c8 OPTIONS (SET p2 'V2', DROP p1);
  \d+ ft1
!                                Foreign table "public.ft1"
!  Column |  Type   | Modifiers |          FDW Options           | Storage  | Description 
! --------+---------+-----------+--------------------------------+----------+-------------
!  c1     | integer | not null  | ("param 1" 'val1')             | plain    | 
!  c2     | text    |           | (param2 'val2', param3 'val3') | extended | 
!  c3     | date    |           |                                | plain    | 
!  c4     | integer |           |                                | plain    | 
!  c6     | integer | not null  |                                | plain    | 
!  c7     | integer |           | (p1 'v1', p2 'v2')             | plain    | 
!  c8     | text    |           | (p2 'V2')                      | extended | 
!  c9     | integer |           |                                | plain    | 
!  c10    | integer |           | (p1 'v1')                      | plain    | 
  Server: s0
  FDW Options: (delimiter ',', quote '"', "be quoted" 'value')
  Has OIDs: no
--- 730,753 ----
  ALTER FOREIGN TABLE ft1 ALTER COLUMN c7 OPTIONS (ADD p1 'v1', ADD p2 'v2'),
                          ALTER COLUMN c8 OPTIONS (ADD p1 'v1', ADD p2 'v2');
  ALTER FOREIGN TABLE ft1 ALTER COLUMN c8 OPTIONS (SET p2 'V2', DROP p1);
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c1 SET STATISTICS 10000;
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c1 SET (n_distinct = 100);
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c1 SET (n_distinct_inherited = 100); -- ERROR
+ ERROR:  cannot set "n_distinct_inherited" for foreign tables
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c8 SET STATISTICS -1;
  \d+ ft1
!                                       Foreign table "public.ft1"
!  Column |  Type   | Modifiers |          FDW Options           | Storage  | Stats target | Description 
! --------+---------+-----------+--------------------------------+----------+--------------+-------------
!  c1     | integer | not null  | ("param 1" 'val1')             | plain    | 10000        | 
!  c2     | text    |           | (param2 'val2', param3 'val3') | extended |              | 
!  c3     | date    |           |                                | plain    |              | 
!  c4     | integer |           |                                | plain    |              | 
!  c6     | integer | not null  |                                | plain    |              | 
!  c7     | integer |           | (p1 'v1', p2 'v2')             | plain    |              | 
!  c8     | text    |           | (p2 'V2')                      | extended |              | 
!  c9     | integer |           |                                | plain    |              | 
!  c10    | integer |           | (p1 'v1')                      | plain    |              | 
  Server: s0
  FDW Options: (delimiter ',', quote '"', "be quoted" 'value')
  Has OIDs: no
diff --git a/src/test/regress/sql/foreign_data.sql b/src/test/regress/sql/foreign_data.sql
index 0c95672..03b5680 100644
*** a/src/test/regress/sql/foreign_data.sql
--- b/src/test/regress/sql/foreign_data.sql
*************** ALTER FOREIGN TABLE ft1 ALTER COLUMN xmi
*** 307,312 ****
--- 307,316 ----
  ALTER FOREIGN TABLE ft1 ALTER COLUMN c7 OPTIONS (ADD p1 'v1', ADD p2 'v2'),
                          ALTER COLUMN c8 OPTIONS (ADD p1 'v1', ADD p2 'v2');
  ALTER FOREIGN TABLE ft1 ALTER COLUMN c8 OPTIONS (SET p2 'V2', DROP p1);
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c1 SET STATISTICS 10000;
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c1 SET (n_distinct = 100);
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c1 SET (n_distinct_inherited = 100); -- ERROR
+ ALTER FOREIGN TABLE ft1 ALTER COLUMN c8 SET STATISTICS -1;
  \d+ ft1
  -- can't change the column type if it's used elsewhere
  CREATE TABLE use_ft1_column_type (x ft1);


  [application/gzip] pgsql_fdw.tar.gz (31.7K, ../4F7D8BBF.4050004@gmail.com/3-pgsql_fdw.tar.gz)
  download

view thread (35+ messages)  latest in thread

Message-ID: <4F7D8BBF.4050004@gmail.com>
Permalink:  ../4F7D8BBF.4050004@gmail.com/
Also on:    postgresql.org/message-id/4F7D8BBF.4050004@gmail.com

copy link · copy postgr.es · copy git am

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: shigeru.hanada@gmail.com, fujita.etsuro@lab.ntt.co.jp
  Subject: Re: WIP: Collecting statistics on CSV file data
  In-Reply-To: <4F7D8BBF.4050004@gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox