Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cZgvG-00024h-ST for pgsql-hackers@arkaria.postgresql.org; Fri, 03 Feb 2017 16:41:18 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1cZgvG-0007A7-CQ for pgsql-hackers@arkaria.postgresql.org; Fri, 03 Feb 2017 16:41:18 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1cZgvF-00079G-4E for pgsql-hackers@postgresql.org; Fri, 03 Feb 2017 16:41:17 +0000 Received: from mx1.redhat.com ([209.132.183.28]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1cZgv7-0007rz-R6 for pgsql-hackers@postgresql.org; Fri, 03 Feb 2017 16:41:15 +0000 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 18EC1C04BD25; Fri, 3 Feb 2017 16:41:09 +0000 (UTC) Received: from localhost.localdomain (vpn-59-183.rdu2.redhat.com [10.10.59.183]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v13Gf7W9002866 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 3 Feb 2017 11:41:07 -0500 Subject: Re: pageinspect: Hash index support To: Robert Haas , Tom Lane References: <4f9c865b-0f79-83a7-a540-1533706c24ee@redhat.com> <408828d9-e1b1-16b0-63fe-f2aeea83bc5c@redhat.com> <17311.1486134714@sss.pgh.pa.us> Cc: Ashutosh Sharma , Mithun Cy , Amit Kapila , Peter Eisentraut , Michael Paquier , Jeff Janes , Alvaro Herrera , PostgreSQL Hackers From: Jesper Pedersen Organization: JBoss, by Red Hat Message-ID: <095699f0-22f2-49fe-672b-b746f3219120@redhat.com> Date: Fri, 3 Feb 2017 11:41:05 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------E14144922737521F982CBC26" X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 03 Feb 2017 16:41:09 +0000 (UTC) X-Pg-Spam-Score: -6.9 (------) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-hackers Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org This is a multi-part message in MIME format. --------------E14144922737521F982CBC26 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hi, On 02/03/2017 11:36 AM, Robert Haas wrote: > On Fri, Feb 3, 2017 at 10:11 AM, Tom Lane wrote: >> BTW, the buildfarm is still crashing on ia64 and sparc64 members. >> I believe this is the same problem addressed in 84ad68d64 for >> pageinspect's GIN functions, to wit, the payload of a "bytea" >> is not maxaligned, so machines that care about alignment won't be >> happy when trying to fetch 64-bit values out of a bytea page image. >> >> Clearly, the fix should be to start using the get_page_from_raw() >> function that Peter introduced in that patch. But it's static in >> ginfuncs.c, which I thought was a mistake at the time, and it's >> proven so now. >> >> contrib/pageinspect actually seems to lack *any* infrastructure >> for sharing functions across modules. It's time to remedy that. >> I propose inventing "pageinspect.h" to hold commonly visible >> declarations, and moving get_page_from_raw() into rawpage.c, >> which seems like a reasonably natural home for it. (Alternatively, >> we could invent a pageinspect.c file to hold utility functions, >> but that might be overkill.) > > No objections. > Attached is v1 of this w/ verify_hash_page() using get_page_from_raw(). Sorry for all the problems. Best regards, Jesper --------------E14144922737521F982CBC26 Content-Type: text/x-patch; name="0001-Add-pageinspect.h_v1.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-Add-pageinspect.h_v1.patch" From d674c58098580ec59f9e1e47255c68b8497178b1 Mon Sep 17 00:00:00 2001 From: jesperpedersen Date: Fri, 3 Feb 2017 11:28:19 -0500 Subject: [PATCH] Add pageinspect.h --- contrib/pageinspect/brinfuncs.c | 1 + contrib/pageinspect/btreefuncs.c | 1 + contrib/pageinspect/fsmfuncs.c | 1 + contrib/pageinspect/ginfuncs.c | 21 +------------- contrib/pageinspect/hashfuncs.c | 3 +- contrib/pageinspect/pageinspect.h | 61 +++++++++++++++++++++++++++++++++++++++ contrib/pageinspect/rawpage.c | 25 ++++++++++++++++ 7 files changed, 92 insertions(+), 21 deletions(-) create mode 100644 contrib/pageinspect/pageinspect.h diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c index 7b877e3..23136d2 100644 --- a/contrib/pageinspect/brinfuncs.c +++ b/contrib/pageinspect/brinfuncs.c @@ -9,6 +9,7 @@ */ #include "postgres.h" +#include "pageinspect.h" #include "access/htup_details.h" #include "access/brin.h" #include "access/brin_internal.h" diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.c index 3f09d5f..e63b5fb 100644 --- a/contrib/pageinspect/btreefuncs.c +++ b/contrib/pageinspect/btreefuncs.c @@ -27,6 +27,7 @@ #include "postgres.h" +#include "pageinspect.h" #include "access/nbtree.h" #include "catalog/namespace.h" #include "catalog/pg_am.h" diff --git a/contrib/pageinspect/fsmfuncs.c b/contrib/pageinspect/fsmfuncs.c index 6541646..372de39 100644 --- a/contrib/pageinspect/fsmfuncs.c +++ b/contrib/pageinspect/fsmfuncs.c @@ -19,6 +19,7 @@ #include "postgres.h" +#include "pageinspect.h" #include "funcapi.h" #include "lib/stringinfo.h" #include "miscadmin.h" diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c index cea77d3..5cfcbcb 100644 --- a/contrib/pageinspect/ginfuncs.c +++ b/contrib/pageinspect/ginfuncs.c @@ -9,6 +9,7 @@ */ #include "postgres.h" +#include "pageinspect.h" #include "access/gin.h" #include "access/gin_private.h" #include "access/htup_details.h" @@ -29,26 +30,6 @@ PG_FUNCTION_INFO_V1(gin_page_opaque_info); PG_FUNCTION_INFO_V1(gin_leafpage_items); -static Page -get_page_from_raw(bytea *raw_page) -{ - int raw_page_size; - Page page; - - raw_page_size = VARSIZE(raw_page) - VARHDRSZ; - if (raw_page_size < BLCKSZ) - ereport(ERROR, - (errcode(ERRCODE_INVALID_PARAMETER_VALUE), - errmsg("input page too small (%d bytes)", raw_page_size))); - - /* make a copy so that the page is properly aligned for struct access */ - page = palloc(raw_page_size); - memcpy(page, VARDATA(raw_page), raw_page_size); - - return page; -} - - Datum gin_metapage_info(PG_FUNCTION_ARGS) { diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.c index 83b4698..09bb9ee 100644 --- a/contrib/pageinspect/hashfuncs.c +++ b/contrib/pageinspect/hashfuncs.c @@ -10,6 +10,7 @@ #include "postgres.h" +#include "pageinspect.h" #include "access/hash.h" #include "access/htup_details.h" #include "catalog/pg_type.h" @@ -67,7 +68,7 @@ verify_hash_page(bytea *raw_page, int flags) errdetail("Expected size %d, got %d", BLCKSZ, raw_page_size))); - page = VARDATA(raw_page); + page = get_page_from_raw(raw_page); if (PageIsNew(page)) ereport(ERROR, diff --git a/contrib/pageinspect/pageinspect.h b/contrib/pageinspect/pageinspect.h new file mode 100644 index 0000000..9ec3aad --- /dev/null +++ b/contrib/pageinspect/pageinspect.h @@ -0,0 +1,61 @@ +/* + * pageinspect.h + * Functions to investigate the content of indexes + * + * Copyright (c) 2017, PostgreSQL Global Development Group + * + * IDENTIFICATION + * contrib/pageinspect/pageinspect.h + */ +#ifndef PAGEINSPECT_H +#define PAGEINSPECT_H + +#include "postgres.h" + +#include "funcapi.h" + +// Raw page functions + +extern Datum get_raw_page(PG_FUNCTION_ARGS); +extern Datum get_raw_page_fork(PG_FUNCTION_ARGS); +extern Page get_page_from_raw(bytea *raw_page); + +extern Datum page_header(PG_FUNCTION_ARGS); + +// FSM functions + +extern Datum fsm_page_contents(PG_FUNCTION_ARGS); + +// Heap functions + +extern Datum heap_page_items(PG_FUNCTION_ARGS); +extern Datum tuple_data_split(PG_FUNCTION_ARGS); + +// B-tree functions + +extern Datum bt_page_stats(PG_FUNCTION_ARGS); +extern Datum bt_page_items(PG_FUNCTION_ARGS); +extern Datum bt_metap(PG_FUNCTION_ARGS); + +// GIN functions + +extern Datum gin_metapage_info(PG_FUNCTION_ARGS); +extern Datum gin_page_opaque_info(PG_FUNCTION_ARGS); +extern Datum gin_leafpage_items(PG_FUNCTION_ARGS); + +// BRIN functions + +extern Datum brin_page_type(PG_FUNCTION_ARGS); +extern Datum brin_page_items(PG_FUNCTION_ARGS); +extern Datum brin_metapage_info(PG_FUNCTION_ARGS); +extern Datum brin_revmap_data(PG_FUNCTION_ARGS); + +// Hash functions + +extern Datum hash_page_type(PG_FUNCTION_ARGS); +extern Datum hash_page_stats(PG_FUNCTION_ARGS); +extern Datum hash_page_items(PG_FUNCTION_ARGS); +extern Datum hash_bitmap_info(PG_FUNCTION_ARGS); +extern Datum hash_metapage_info(PG_FUNCTION_ARGS); + +#endif diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c index a2ac078..991ae84 100644 --- a/contrib/pageinspect/rawpage.c +++ b/contrib/pageinspect/rawpage.c @@ -15,6 +15,7 @@ #include "postgres.h" +#include "pageinspect.h" #include "access/htup_details.h" #include "catalog/catalog.h" #include "catalog/namespace.h" @@ -86,6 +87,30 @@ get_raw_page_fork(PG_FUNCTION_ARGS) } /* + * get_page_from_raw + * + * Get a page + */ +Page +get_page_from_raw(bytea *raw_page) +{ + int raw_page_size; + Page page; + + raw_page_size = VARSIZE(raw_page) - VARHDRSZ; + if (raw_page_size < BLCKSZ) + ereport(ERROR, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("input page too small (%d bytes)", raw_page_size))); + + /* make a copy so that the page is properly aligned for struct access */ + page = palloc(raw_page_size); + memcpy(page, VARDATA(raw_page), raw_page_size); + + return page; +} + +/* * workhorse */ static bytea * -- 2.7.4 --------------E14144922737521F982CBC26 Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 8bit MIME-Version: 1.0 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers --------------E14144922737521F982CBC26--