Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hBvI1-0007xq-Se for pgsql-hackers@arkaria.postgresql.org; Thu, 04 Apr 2019 05:51:54 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1hBvHy-0001uh-PG for pgsql-hackers@arkaria.postgresql.org; Thu, 04 Apr 2019 05:51:50 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hBvHy-0001uV-BT for pgsql-hackers@lists.postgresql.org; Thu, 04 Apr 2019 05:51:50 +0000 Received: from mail-it1-x141.google.com ([2607:f8b0:4864:20::141]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hBvHr-0007js-RU for pgsql-hackers@postgresql.org; Thu, 04 Apr 2019 05:51:47 +0000 Received: by mail-it1-x141.google.com with SMTP id x132so1855685itf.2 for ; Wed, 03 Apr 2019 22:51:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telsasoft-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=ujH8d6mWZeN/to5LzQaPfJc1kywGL5vWx3VmGohvWx8=; b=X2nRt28J6C6jzMyHvpq3p0+sVWmrgBK45ns/P19rypKVctMDlt1pRo2JHD89F37tpC R39tRsnnbMItMsbpKe5EKQdfpynehuaDCM0Hrs1ljemjyqpgayRl/4CI6ZuBfuC47XKz 85n7sto+b/2P2uZKl2/xXlz0lABlomiZW3y3yowvwYq+D0vJnh4fXkoNoRxOlUbJRfw3 w9coDqiVv83vVG81vHen7+w3aUIdUrqacYPwqasx5AkoNxgmNeNjXwLg4MaHP+kVRwzj Bl98XqEBipqAqCL6T9818rYAlc6Dud8kDpBw7kBzCWOuAJDEgTkmkdmQUioSwR8vfEei VVtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=ujH8d6mWZeN/to5LzQaPfJc1kywGL5vWx3VmGohvWx8=; b=JVOFZdRRONaQJ1gVBMLtMp5mP4ibdDiLlLIDuIbnv6fG6kVYueJ3kMwKmkk60ERWKZ IrTmlZDOsw6hCZmCNh1IMW1cgsLrC/raPrtC1nSuQutQDvmiCbcUjZf0s/EgN+nfYHOx d5JZRl7qnxgLLgju1Z6xpF140PglCU/wUoAVQLXCEBGmob9eXktMKqCwg8oEcXidXb7s TijQWnaMAnpnsEkCPmplKo2nrc9n1oMTRw6JuNZXCODyg93R331yFN5265N0rcBWHT5n j6vKsWVC+7yDnFUoEZooI28pmsZqnPm69uk81osgcFPO6Ezlp8jLS+cHMD0R8vF08nfM hPjw== X-Gm-Message-State: APjAAAWcnTPOSdz+HnBV1/VEofgFHGIavt3armv4c7PFba+WsRTQBzxN qexnhsq6xiGKF33t58HuBHeKUg== X-Google-Smtp-Source: APXvYqzeq33sf50lJIhNhZqWSAuIGYzmrvkKyZY3rj8bmRKCqcWMTttYTwRsCBowNJHNolJX1AUfoA== X-Received: by 2002:a24:43d1:: with SMTP id s200mr3586094itb.155.1554357101100; Wed, 03 Apr 2019 22:51:41 -0700 (PDT) Received: from pryzbyj (charmander.telsasoft.com. [50.244.222.1]) by smtp.gmail.com with ESMTPSA id o141sm2089671ito.21.2019.04.03.22.51.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Apr 2019 22:51:39 -0700 (PDT) Received: by pryzbyj (Postfix, from userid 1000) id 843E9800FD3; Thu, 4 Apr 2019 00:51:38 -0500 (CDT) Date: Thu, 4 Apr 2019 00:51:38 -0500 From: Justin Pryzby To: Andres Freund Cc: Haribabu Kommi , Kyotaro HORIGUCHI , Alvaro Herrera , David Rowley , Robert Haas , Asim R P , pgsql-hackers@postgresql.org, Ashwin Agrawal , Dmitry Dolgov <9erthalion6@gmail.com>, Alexander Korotkov , Heikki Linnakangas Subject: Re: Pluggable Storage - Andres's take Message-ID: <20190404055138.GA24864@telsasoft.com> References: <20190309031825.zlgqs3tg424g3lkf@alap3.anarazel.de> <20190312.172757.17614556.horiguchi.kyotaro@lab.ntt.co.jp> <20190401231810.arpvmhw66bwsyobj@alap3.anarazel.de> <20190402005352.64hk76gddvs36uwl@alap3.anarazel.de> <20190402215743.cfiwzlk2szwq4jp3@alap3.anarazel.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="ibTvN161/egqYuK8" Content-Disposition: inline In-Reply-To: <20190402215743.cfiwzlk2szwq4jp3@alap3.anarazel.de> User-Agent: Mutt/1.5.23 (2014-03-12) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --ibTvN161/egqYuK8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I reviewed new docs for $SUBJECT. Find attached proposed changes. There's one XXX item I'm unsure what it's intended to say. Justin From a3d290bf67af2a34e44cd6c160daf552b56a13b5 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Thu, 4 Apr 2019 00:48:09 -0500 Subject: [PATCH v1] Fine tune documentation for tableam Added at commit b73c3a11963c8bb783993cfffabb09f558f86e37 --- doc/src/sgml/catalogs.sgml | 2 +- doc/src/sgml/config.sgml | 4 ++-- doc/src/sgml/ref/select_into.sgml | 6 +++--- doc/src/sgml/storage.sgml | 17 ++++++++------- doc/src/sgml/tableam.sgml | 44 ++++++++++++++++++++------------------- 5 files changed, 38 insertions(+), 35 deletions(-) diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index 58c8c96..40ddec4 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -587,7 +587,7 @@ The catalog pg_am stores information about relation access methods. There is one row for each access method supported by the system. - Currently, only table and indexes have access methods. The requirements for table + Currently, only tables and indexes have access methods. The requirements for table and index access methods are discussed in detail in and respectively. diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 4a9a1e8..90b478d 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -7306,8 +7306,8 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; This parameter specifies the default table access method to use when creating tables or materialized views if the CREATE command does not explicitly specify an access method, or when - SELECT ... INTO is used, which does not allow to - specify a table access method. The default is heap. + SELECT ... INTO is used, which does not allow + specification of a table access method. The default is heap. diff --git a/doc/src/sgml/ref/select_into.sgml b/doc/src/sgml/ref/select_into.sgml index 17bed24..1443d79 100644 --- a/doc/src/sgml/ref/select_into.sgml +++ b/doc/src/sgml/ref/select_into.sgml @@ -106,11 +106,11 @@ SELECT [ ALL | DISTINCT [ ON ( expression - In contrast to CREATE TABLE AS SELECT - INTO does not allow to specify properties like a table's access + In contrast to CREATE TABLE AS, SELECT + INTO does not allow specification of properties like a table's access method with or the table's tablespace with . Use if necessary. Therefore the default table + linkend="sql-createtableas"/> if necessary. Therefore, the default table access method is chosen for the new table. See for more information. diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml index 62333e3..5dfca1b 100644 --- a/doc/src/sgml/storage.sgml +++ b/doc/src/sgml/storage.sgml @@ -189,11 +189,11 @@ there. - Note that the following sections describe the way the builtin + Note that the following sections describe the behavior of the builtin heap table access method, - and the builtin index access methods work. Due - to the extensible nature of PostgreSQL other types - of access method might work similar or not. + and the builtin index access methods. Due + to the extensible nature of PostgreSQL, other + access methods might work differently. @@ -703,11 +703,12 @@ erased (they will be recreated automatically as needed). This section provides an overview of the page format used within PostgreSQL tables and indexes. - Actually, neither table nor index access methods need not use this page - format. All the existing index methods do use this basic format, but the + Actually, use of this page format is not required for either table or index + access methods. + The heap table access method always uses this format. + All the existing index methods also use the basic format, but the data kept on index metapages usually doesn't follow the item layout - rules. The heap table access method also always uses - this format. + rules. Sequences and TOAST tables are formatted just like a regular table. diff --git a/doc/src/sgml/tableam.sgml b/doc/src/sgml/tableam.sgml index 8d9bfd8..0a89935 100644 --- a/doc/src/sgml/tableam.sgml +++ b/doc/src/sgml/tableam.sgml @@ -48,54 +48,56 @@ callbacks and their behavior is defined in the TableAmRoutine structure (with comments inside the struct defining the requirements for callbacks). Most callbacks have - wrapper functions, which are documented for the point of view of a user, - rather than an implementor, of the table access method. For details, + wrapper functions, which are documented from the point of view of a user + (rather than an implementor) of the table access method. For details, please refer to the src/include/access/tableam.h file. - To implement a access method, an implementor will typically need to - implement a AM specific type of tuple table slot (see + To implement an access method, an implementor will typically need to + implement an AM-specific type of tuple table slot (see - src/include/executor/tuptable.h) which allows + src/include/executor/tuptable.h), which allows code outside the access method to hold references to tuples of the AM, and to access the columns of the tuple. - Currently the the way an AM actually stores data is fairly - unconstrained. It is e.g. possible to use postgres' shared buffer cache, - but not required. In case shared buffers are used, it likely makes to - postgres' standard page layout described in . One fairly large constraint of the table access method API is that, currently, if the AM wants to support modifications and/or indexes, it is - necessary that each tuple has a tuple identifier (TID) + necessary for each tuple to have a tuple identifier (TID) consisting of a block number and an item number (see also ). It is not strictly necessary that the - sub-parts of TIDs have the same meaning they e.g. have + sub-parts of TIDs have the same meaning as used for heap, but if bitmap scan support is desired (it is optional), the block number needs to provide locality. - For crash safety an AM can use postgres' WAL, or a custom approach can be - implemented. If WAL is chosen, either Generic WAL Records can be used — which - implies higher WAL volume but is easy, or a new type of - WAL records can be implemented — but that - currently requires modifications of core code (namely modifying + For crash safety, an AM can use postgres' WAL, or a custom implementation. + If WAL is chosen, either Generic WAL Records can be used, + or a new type of WAL records can be implemented. + Generic WAL Records are easy, but imply higher WAL volume. + Implementation of a new type of WAL record + currently requires modifications to core code (specifically, src/include/access/rmgrlist.h). To implement transactional support in a manner that allows different table - access methods be accessed within a single transaction, it likely is + access methods be accessed within a single transaction, it's likely necessary to closely integrate with the machinery in src/backend/access/transam/xlog.c. @@ -103,8 +105,8 @@ Any developer of a new table access method can refer to the existing heap implementation present in - src/backend/heap/heapam_handler.c for more details of - how it is implemented. + src/backend/heap/heapam_handler.c for details of + its implementation. -- 2.1.4 --ibTvN161/egqYuK8 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v1-0001-Fine-tune-documentation-for-tableam.patch" From a3d290bf67af2a34e44cd6c160daf552b56a13b5 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Thu, 4 Apr 2019 00:48:09 -0500 Subject: [PATCH v1] Fine tune documentation for tableam Added at commit b73c3a11963c8bb783993cfffabb09f558f86e37 --- doc/src/sgml/catalogs.sgml | 2 +- doc/src/sgml/config.sgml | 4 ++-- doc/src/sgml/ref/select_into.sgml | 6 +++--- doc/src/sgml/storage.sgml | 17 ++++++++------- doc/src/sgml/tableam.sgml | 44 ++++++++++++++++++++------------------- 5 files changed, 38 insertions(+), 35 deletions(-) diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index 58c8c96..40ddec4 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -587,7 +587,7 @@ The catalog pg_am stores information about relation access methods. There is one row for each access method supported by the system. - Currently, only table and indexes have access methods. The requirements for table + Currently, only tables and indexes have access methods. The requirements for table and index access methods are discussed in detail in and respectively. diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 4a9a1e8..90b478d 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -7306,8 +7306,8 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv; This parameter specifies the default table access method to use when creating tables or materialized views if the CREATE command does not explicitly specify an access method, or when - SELECT ... INTO is used, which does not allow to - specify a table access method. The default is heap. + SELECT ... INTO is used, which does not allow + specification of a table access method. The default is heap. diff --git a/doc/src/sgml/ref/select_into.sgml b/doc/src/sgml/ref/select_into.sgml index 17bed24..1443d79 100644 --- a/doc/src/sgml/ref/select_into.sgml +++ b/doc/src/sgml/ref/select_into.sgml @@ -106,11 +106,11 @@ SELECT [ ALL | DISTINCT [ ON ( expression - In contrast to CREATE TABLE AS SELECT - INTO does not allow to specify properties like a table's access + In contrast to CREATE TABLE AS, SELECT + INTO does not allow specification of properties like a table's access method with or the table's tablespace with . Use if necessary. Therefore the default table + linkend="sql-createtableas"/> if necessary. Therefore, the default table access method is chosen for the new table. See for more information. diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml index 62333e3..5dfca1b 100644 --- a/doc/src/sgml/storage.sgml +++ b/doc/src/sgml/storage.sgml @@ -189,11 +189,11 @@ there. - Note that the following sections describe the way the builtin + Note that the following sections describe the behavior of the builtin heap table access method, - and the builtin index access methods work. Due - to the extensible nature of PostgreSQL other types - of access method might work similar or not. + and the builtin index access methods. Due + to the extensible nature of PostgreSQL, other + access methods might work differently. @@ -703,11 +703,12 @@ erased (they will be recreated automatically as needed). This section provides an overview of the page format used within PostgreSQL tables and indexes. - Actually, neither table nor index access methods need not use this page - format. All the existing index methods do use this basic format, but the + Actually, use of this page format is not required for either table or index + access methods. + The heap table access method always uses this format. + All the existing index methods also use the basic format, but the data kept on index metapages usually doesn't follow the item layout - rules. The heap table access method also always uses - this format. + rules. Sequences and TOAST tables are formatted just like a regular table. diff --git a/doc/src/sgml/tableam.sgml b/doc/src/sgml/tableam.sgml index 8d9bfd8..0a89935 100644 --- a/doc/src/sgml/tableam.sgml +++ b/doc/src/sgml/tableam.sgml @@ -48,54 +48,56 @@ callbacks and their behavior is defined in the TableAmRoutine structure (with comments inside the struct defining the requirements for callbacks). Most callbacks have - wrapper functions, which are documented for the point of view of a user, - rather than an implementor, of the table access method. For details, + wrapper functions, which are documented from the point of view of a user + (rather than an implementor) of the table access method. For details, please refer to the src/include/access/tableam.h file. - To implement a access method, an implementor will typically need to - implement a AM specific type of tuple table slot (see + To implement an access method, an implementor will typically need to + implement an AM-specific type of tuple table slot (see - src/include/executor/tuptable.h) which allows + src/include/executor/tuptable.h), which allows code outside the access method to hold references to tuples of the AM, and to access the columns of the tuple. - Currently the the way an AM actually stores data is fairly - unconstrained. It is e.g. possible to use postgres' shared buffer cache, - but not required. In case shared buffers are used, it likely makes to - postgres' standard page layout described in . One fairly large constraint of the table access method API is that, currently, if the AM wants to support modifications and/or indexes, it is - necessary that each tuple has a tuple identifier (TID) + necessary for each tuple to have a tuple identifier (TID) consisting of a block number and an item number (see also ). It is not strictly necessary that the - sub-parts of TIDs have the same meaning they e.g. have + sub-parts of TIDs have the same meaning as used for heap, but if bitmap scan support is desired (it is optional), the block number needs to provide locality. - For crash safety an AM can use postgres' WAL, or a custom approach can be - implemented. If WAL is chosen, either Generic WAL Records can be used — which - implies higher WAL volume but is easy, or a new type of - WAL records can be implemented — but that - currently requires modifications of core code (namely modifying + For crash safety, an AM can use postgres' WAL, or a custom implementation. + If WAL is chosen, either Generic WAL Records can be used, + or a new type of WAL records can be implemented. + Generic WAL Records are easy, but imply higher WAL volume. + Implementation of a new type of WAL record + currently requires modifications to core code (specifically, src/include/access/rmgrlist.h). To implement transactional support in a manner that allows different table - access methods be accessed within a single transaction, it likely is + access methods be accessed within a single transaction, it's likely necessary to closely integrate with the machinery in src/backend/access/transam/xlog.c. @@ -103,8 +105,8 @@ Any developer of a new table access method can refer to the existing heap implementation present in - src/backend/heap/heapam_handler.c for more details of - how it is implemented. + src/backend/heap/heapam_handler.c for details of + its implementation. -- 2.1.4 --ibTvN161/egqYuK8--