pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Michael Paquier <michael@paquier.xyz>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Move pg_attribute.attcompression to earlier in struct for reduced size?
Date: Fri, 4 Jun 2021 19:21:11 -0400
Message-ID: <202106042321.6jx54yliy2l6@alvherre.pgsql> (raw)
In-Reply-To: <449921.1622847060@sss.pgh.pa.us>

On 2021-Jun-04, Tom Lane wrote:

> Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

> > Now, while this patch does seem to work correctly, it raises a number of
> > weird cpluspluscheck warnings, which I think are attributable to the
> > new macro definitions.  I didn't look into it closely, but I suppose it
> > should be fixable given sufficient effort:
> 
> Didn't test, but the first one is certainly fixable by adding a cast,
> and I guess the others might be as well.

I get no warnings with this one.  I'm a bit wary of leaving
VARDATA_COMPRESSED_GET_EXTSIZE unchanged, but at least nothing in this
patch requires a cast there.

-- 
Álvaro Herrera       Valdivia, Chile

Attachments:

v2-0001-Reduce-overhead-of-TOAST-recompression-on-table-r.patch (text/x-diff, 4.3K) download From d1a0b624e316dcd50aabaf21c4ab4de19b15b604 Mon Sep 17 00:00:00 2001 From: Alvaro Herrera <alvherre@alvh.no-ip.org> Date: Fri, 4 Jun 2021 19:09:30 -0400 Subject: [PATCH v2] Reduce overhead of TOAST recompression on table rewrite --- src/backend/access/common/toast_compression.c | 30 --------------- src/include/access/toast_compression.h | 37 ++++++++++++++++++- src/include/postgres.h | 6 +-- 3 files changed, 39 insertions(+), 34 deletions(-) diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c index 9e9d4457ac..38688e7cfc 100644 --- a/src/backend/access/common/toast_compression.c +++ b/src/backend/access/common/toast_compression.c @@ -247,36 +247,6 @@ lz4_decompress_datum_slice(const struct varlena *value, int32 slicelength) #endif } -/* - * Extract compression ID from a varlena. - * - * Returns TOAST_INVALID_COMPRESSION_ID if the varlena is not compressed. - */ -ToastCompressionId -toast_get_compression_id(struct varlena *attr) -{ - ToastCompressionId cmid = TOAST_INVALID_COMPRESSION_ID; - - /* - * If it is stored externally then fetch the compression method id from - * the external toast pointer. If compressed inline, fetch it from the - * toast compression header. - */ - if (VARATT_IS_EXTERNAL_ONDISK(attr)) - { - struct varatt_external toast_pointer; - - VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr); - - if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer)) - cmid = VARATT_EXTERNAL_GET_COMPRESS_METHOD(toast_pointer); - } - else if (VARATT_IS_COMPRESSED(attr)) - cmid = VARDATA_COMPRESSED_GET_COMPRESS_METHOD(attr); - - return cmid; -} - /* * CompressionNameToMethod - Get compression method from compression name * diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h index 9e2c1cbe1a..8756de77d8 100644 --- a/src/include/access/toast_compression.h +++ b/src/include/access/toast_compression.h @@ -13,6 +13,8 @@ #ifndef TOAST_COMPRESSION_H #define TOAST_COMPRESSION_H +#include "access/detoast.h" + /* * GUC support. * @@ -78,8 +80,41 @@ extern struct varlena *lz4_decompress_datum_slice(const struct varlena *value, int32 slicelength); /* other stuff */ -extern ToastCompressionId toast_get_compression_id(struct varlena *attr); extern char CompressionNameToMethod(const char *compression); extern const char *GetCompressionMethodName(char method); + +/* + * Extract compression ID from a varlena. + * + * Returns TOAST_INVALID_COMPRESSION_ID if the varlena is not compressed. + */ +static inline ToastCompressionId +toast_get_compression_id(struct varlena *attr) +{ + ToastCompressionId cmid = TOAST_INVALID_COMPRESSION_ID; + + /* + * If it is stored externally then fetch the compression method id from + * the external toast pointer. If compressed inline, fetch it from the + * toast compression header. + */ + if (VARATT_IS_EXTERNAL_ONDISK(attr)) + { + struct varatt_external toast_pointer; + + VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr); + + if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer)) + cmid = VARATT_EXTERNAL_GET_COMPRESS_METHOD(toast_pointer); + } + else if (VARATT_IS_COMPRESSED(attr)) + cmid = VARDATA_COMPRESSED_GET_COMPRESS_METHOD(attr); + + return cmid; +} + + + + #endif /* TOAST_COMPRESSION_H */ diff --git a/src/include/postgres.h b/src/include/postgres.h index 8e02f91108..7437f4dcf3 100644 --- a/src/include/postgres.h +++ b/src/include/postgres.h @@ -365,13 +365,13 @@ typedef struct #define VARDATA_COMPRESSED_GET_EXTSIZE(PTR) \ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_EXTSIZE_MASK) #define VARDATA_COMPRESSED_GET_COMPRESS_METHOD(PTR) \ - (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_EXTSIZE_BITS) + ((ToastCompressionId) (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_EXTSIZE_BITS)) /* Same, when working directly with a struct varatt_external */ #define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \ - ((toast_pointer).va_extinfo & VARLENA_EXTSIZE_MASK) + ((int32) ((toast_pointer).va_extinfo & VARLENA_EXTSIZE_MASK)) #define VARATT_EXTERNAL_GET_COMPRESS_METHOD(toast_pointer) \ - ((toast_pointer).va_extinfo >> VARLENA_EXTSIZE_BITS) + ((ToastCompressionId) ((toast_pointer).va_extinfo >> VARLENA_EXTSIZE_BITS)) #define VARATT_EXTERNAL_SET_SIZE_AND_COMPRESS_METHOD(toast_pointer, len, cm) \ do { \ -- 2.20.1

view thread (73+ messages)  latest in thread

Message-ID: <202106042321.6jx54yliy2l6@alvherre.pgsql>
Permalink:  ../202106042321.6jx54yliy2l6@alvherre.pgsql/
Also on:    postgresql.org/message-id/202106042321.6jx54yliy2l6@alvherre.pgsql

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-hackers@postgresql.org
  Cc: alvherre@alvh.no-ip.org, tgl@sss.pgh.pa.us, michael@paquier.xyz, robertmhaas@gmail.com, dilipbalaut@gmail.com, andres@anarazel.de
  Subject: Re: Move pg_attribute.attcompression to earlier in struct for reduced size?
  In-Reply-To: <202106042321.6jx54yliy2l6@alvherre.pgsql>
  References: <449921.1622847060@sss.pgh.pa.us> <202106042321.6jx54yliy2l6@alvherre.pgsql>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox