From: Jesper Pedersen <jesper.pedersen@redhat.com>
To: Robert Haas <robertmhaas@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Ashutosh Sharma <ashu.coek88@gmail.com>
Cc: Mithun Cy <mithun.cy@enterprisedb.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: Michael Paquier <michael.paquier@gmail.com>
Cc: Jeff Janes <jeff.janes@gmail.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: pageinspect: Hash index support
Date: Fri, 3 Feb 2017 11:41:05 -0500
Message-ID: <095699f0-22f2-49fe-672b-b746f3219120@redhat.com> (raw)
In-Reply-To: <CA+Tgmob_0UZNbjr53LGuWs-XiezBo5P6ar=Wu2dvxvU=PPOg3A@mail.gmail.com>
References: <e2ac6c58-b93f-9dd9-f4e6-d6d30add7fdf@redhat.com>
<d92c25dc-a42d-d56e-5228-cc79b424fe99@redhat.com>
<CAA4eK1JDXuOUsx-x0FzOJJGX6Xa4jOL80y62ScZLqdEOuGBXMA@mail.gmail.com>
<CAE9k0PnA=CnkaBCYfkL9jMJ5yNm5nEad=Cx2x8ED8Yo2j8+pzA@mail.gmail.com>
<4f9c865b-0f79-83a7-a540-1533706c24ee@redhat.com>
<CAE9k0PkewVp0HzkaiztjiUma8tsZLMi+Gx==4Wszj3tR8A0acA@mail.gmail.com>
<CAD__OugG-LRWN-dFpdwr62v9f4HWDRtAdwzHXVfNAU_fSR-pRw@mail.gmail.com>
<CA+TgmoZUjrVy52TUU3b_Q5L6jcrt7w5R4qFwMXdeCuKQBmYWqQ@mail.gmail.com>
<CAE9k0Pke046HKYfuJGcCtP77NyHrun7hBV-v20a0TW4CUg4H+A@mail.gmail.com>
<CA+TgmoaTRh4B75CguW=pWLjJ=LLBBH-=Anz6pcPFn5jx_VQPQQ@mail.gmail.com>
<CAE9k0Pnkx0mxAejDX6S6EBaNAgv=Fj9DoEx7B0vy=d6kmzcNzQ@mail.gmail.com>
<CA+Tgmoa6cKSV26d_XMwqw2Hwh_hk5wZpuVJg18WTVQ-L6vr+Aw@mail.gmail.com>
<408828d9-e1b1-16b0-63fe-f2aeea83bc5c@redhat.com>
<b2c7d6f0-edc3-6fce-cbbe-82ca277e2f22@redhat.com>
<17311.1486134714@sss.pgh.pa.us>
<CA+Tgmob_0UZNbjr53LGuWs-XiezBo5P6ar=Wu2dvxvU=PPOg3A@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
Hi,
On 02/03/2017 11:36 AM, Robert Haas wrote:
> On Fri, Feb 3, 2017 at 10:11 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> BTW, the buildfarm is still crashing on ia64 and sparc64 members.
>> I believe this is the same problem addressed in 84ad68d64 for
>> pageinspect's GIN functions, to wit, the payload of a "bytea"
>> is not maxaligned, so machines that care about alignment won't be
>> happy when trying to fetch 64-bit values out of a bytea page image.
>>
>> Clearly, the fix should be to start using the get_page_from_raw()
>> function that Peter introduced in that patch. But it's static in
>> ginfuncs.c, which I thought was a mistake at the time, and it's
>> proven so now.
>>
>> contrib/pageinspect actually seems to lack *any* infrastructure
>> for sharing functions across modules. It's time to remedy that.
>> I propose inventing "pageinspect.h" to hold commonly visible
>> declarations, and moving get_page_from_raw() into rawpage.c,
>> which seems like a reasonably natural home for it. (Alternatively,
>> we could invent a pageinspect.c file to hold utility functions,
>> but that might be overkill.)
>
> No objections.
>
Attached is v1 of this w/ verify_hash_page() using get_page_from_raw().
Sorry for all the problems.
Best regards,
Jesper
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackersAttachments:0001-Add-pageinspect.h_v1.patch (text/x-patch, 5.9K) download
From d674c58098580ec59f9e1e47255c68b8497178b1 Mon Sep 17 00:00:00 2001
From: jesperpedersen <jesper.pedersen@redhat.com>
Date: Fri, 3 Feb 2017 11:28:19 -0500
Subject: [PATCH] Add pageinspect.h
---
contrib/pageinspect/brinfuncs.c | 1 +
contrib/pageinspect/btreefuncs.c | 1 +
contrib/pageinspect/fsmfuncs.c | 1 +
contrib/pageinspect/ginfuncs.c | 21 +-------------
contrib/pageinspect/hashfuncs.c | 3 +-
contrib/pageinspect/pageinspect.h | 61 +++++++++++++++++++++++++++++++++++++++
contrib/pageinspect/rawpage.c | 25 ++++++++++++++++
7 files changed, 92 insertions(+), 21 deletions(-)
create mode 100644 contrib/pageinspect/pageinspect.h
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.cindex 7b877e3..23136d2 100644--- a/contrib/pageinspect/brinfuncs.c+++ b/contrib/pageinspect/brinfuncs.c@@ -9,6 +9,7 @@
*/
#include "postgres.h"
+#include "pageinspect.h"
#include "access/htup_details.h"
#include "access/brin.h"
#include "access/brin_internal.h"
diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.cindex 3f09d5f..e63b5fb 100644--- a/contrib/pageinspect/btreefuncs.c+++ b/contrib/pageinspect/btreefuncs.c@@ -27,6 +27,7 @@
#include "postgres.h"
+#include "pageinspect.h"
#include "access/nbtree.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
diff --git a/contrib/pageinspect/fsmfuncs.c b/contrib/pageinspect/fsmfuncs.cindex 6541646..372de39 100644--- a/contrib/pageinspect/fsmfuncs.c+++ b/contrib/pageinspect/fsmfuncs.c@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "pageinspect.h"
#include "funcapi.h"
#include "lib/stringinfo.h"
#include "miscadmin.h"
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.cindex cea77d3..5cfcbcb 100644--- a/contrib/pageinspect/ginfuncs.c+++ b/contrib/pageinspect/ginfuncs.c@@ -9,6 +9,7 @@
*/
#include "postgres.h"
+#include "pageinspect.h"
#include "access/gin.h"
#include "access/gin_private.h"
#include "access/htup_details.h"
@@ -29,26 +30,6 @@ PG_FUNCTION_INFO_V1(gin_page_opaque_info);
PG_FUNCTION_INFO_V1(gin_leafpage_items);
-static Page-get_page_from_raw(bytea *raw_page)-{- int raw_page_size;- Page page;-- raw_page_size = VARSIZE(raw_page) - VARHDRSZ;- if (raw_page_size < BLCKSZ)- ereport(ERROR,- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),- errmsg("input page too small (%d bytes)", raw_page_size)));-- /* make a copy so that the page is properly aligned for struct access */- page = palloc(raw_page_size);- memcpy(page, VARDATA(raw_page), raw_page_size);-- return page;-}--
Datum
gin_metapage_info(PG_FUNCTION_ARGS)
{
diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.cindex 83b4698..09bb9ee 100644--- a/contrib/pageinspect/hashfuncs.c+++ b/contrib/pageinspect/hashfuncs.c@@ -10,6 +10,7 @@
#include "postgres.h"
+#include "pageinspect.h"
#include "access/hash.h"
#include "access/htup_details.h"
#include "catalog/pg_type.h"
@@ -67,7 +68,7 @@ verify_hash_page(bytea *raw_page, int flags)
errdetail("Expected size %d, got %d",
BLCKSZ, raw_page_size)));
- page = VARDATA(raw_page);+ page = get_page_from_raw(raw_page);
if (PageIsNew(page))
ereport(ERROR,
diff --git a/contrib/pageinspect/pageinspect.h b/contrib/pageinspect/pageinspect.hnew file mode 100644index 0000000..9ec3aad--- /dev/null+++ b/contrib/pageinspect/pageinspect.h@@ -0,0 +1,61 @@+/*+ * pageinspect.h+ * Functions to investigate the content of indexes+ *+ * Copyright (c) 2017, PostgreSQL Global Development Group+ *+ * IDENTIFICATION+ * contrib/pageinspect/pageinspect.h+ */+#ifndef PAGEINSPECT_H+#define PAGEINSPECT_H++#include "postgres.h"++#include "funcapi.h"++// Raw page functions++extern Datum get_raw_page(PG_FUNCTION_ARGS);+extern Datum get_raw_page_fork(PG_FUNCTION_ARGS);+extern Page get_page_from_raw(bytea *raw_page);++extern Datum page_header(PG_FUNCTION_ARGS);++// FSM functions++extern Datum fsm_page_contents(PG_FUNCTION_ARGS);++// Heap functions++extern Datum heap_page_items(PG_FUNCTION_ARGS);+extern Datum tuple_data_split(PG_FUNCTION_ARGS);++// B-tree functions++extern Datum bt_page_stats(PG_FUNCTION_ARGS);+extern Datum bt_page_items(PG_FUNCTION_ARGS);+extern Datum bt_metap(PG_FUNCTION_ARGS);++// GIN functions++extern Datum gin_metapage_info(PG_FUNCTION_ARGS);+extern Datum gin_page_opaque_info(PG_FUNCTION_ARGS);+extern Datum gin_leafpage_items(PG_FUNCTION_ARGS);++// BRIN functions++extern Datum brin_page_type(PG_FUNCTION_ARGS);+extern Datum brin_page_items(PG_FUNCTION_ARGS);+extern Datum brin_metapage_info(PG_FUNCTION_ARGS);+extern Datum brin_revmap_data(PG_FUNCTION_ARGS);++// Hash functions++extern Datum hash_page_type(PG_FUNCTION_ARGS);+extern Datum hash_page_stats(PG_FUNCTION_ARGS);+extern Datum hash_page_items(PG_FUNCTION_ARGS);+extern Datum hash_bitmap_info(PG_FUNCTION_ARGS);+extern Datum hash_metapage_info(PG_FUNCTION_ARGS);++#endifdiff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.cindex a2ac078..991ae84 100644--- a/contrib/pageinspect/rawpage.c+++ b/contrib/pageinspect/rawpage.c@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "pageinspect.h"
#include "access/htup_details.h"
#include "catalog/catalog.h"
#include "catalog/namespace.h"
@@ -86,6 +87,30 @@ get_raw_page_fork(PG_FUNCTION_ARGS)
}
/*
+ * get_page_from_raw+ *+ * Get a page+ */+Page+get_page_from_raw(bytea *raw_page)+{+ int raw_page_size;+ Page page;++ raw_page_size = VARSIZE(raw_page) - VARHDRSZ;+ if (raw_page_size < BLCKSZ)+ ereport(ERROR,+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),+ errmsg("input page too small (%d bytes)", raw_page_size)));++ /* make a copy so that the page is properly aligned for struct access */+ page = palloc(raw_page_size);+ memcpy(page, VARDATA(raw_page), raw_page_size);++ return page;+}++/*
* workhorse
*/
static bytea *
--
2.7.4
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: jesper.pedersen@redhat.com, robertmhaas@gmail.com, tgl@sss.pgh.pa.us, ashu.coek88@gmail.com, mithun.cy@enterprisedb.com, amit.kapila16@gmail.com, peter.eisentraut@2ndquadrant.com, michael.paquier@gmail.com, jeff.janes@gmail.com, alvherre@2ndquadrant.com
Subject: Re: pageinspect: Hash index support
In-Reply-To: <095699f0-22f2-49fe-672b-b746f3219120@redhat.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox