public inbox for [email protected]help / color / mirror / Atom feed
[PATCH v24 3/7] Add default_toast_compression GUC 2+ messages / 2 participants [nested] [flat]
* [PATCH v24 3/7] Add default_toast_compression GUC @ 2021-01-30 03:37 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: Justin Pryzby @ 2021-01-30 03:37 UTC (permalink / raw) --- src/backend/access/common/tupdesc.c | 2 +- src/backend/bootstrap/bootstrap.c | 3 +- src/backend/commands/amcmds.c | 143 +++++++++++++++++++++++++++- src/backend/commands/tablecmds.c | 4 +- src/backend/utils/init/postinit.c | 4 + src/backend/utils/misc/guc.c | 12 +++ src/include/access/amapi.h | 2 + src/include/access/compressamapi.h | 12 ++- 8 files changed, 176 insertions(+), 6 deletions(-) diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c index ca26fab487..7afaea000b 100644 --- a/src/backend/access/common/tupdesc.c +++ b/src/backend/access/common/tupdesc.c @@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc, att->attcollation = typeForm->typcollation; if (IsStorageCompressible(typeForm->typstorage)) - att->attcompression = DefaultCompressionOid; + att->attcompression = GetDefaultToastCompression(); else att->attcompression = InvalidOid; diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c index 9b451eaa71..ec3376cf8a 100644 --- a/src/backend/bootstrap/bootstrap.c +++ b/src/backend/bootstrap/bootstrap.c @@ -732,8 +732,9 @@ DefineAttr(char *name, char *type, int attnum, int nullness) attrtypes[attnum]->attcacheoff = -1; attrtypes[attnum]->atttypmod = -1; attrtypes[attnum]->attislocal = true; + if (IsStorageCompressible(attrtypes[attnum]->attstorage)) - attrtypes[attnum]->attcompression = DefaultCompressionOid; + attrtypes[attnum]->attcompression = GetDefaultToastCompression(); else attrtypes[attnum]->attcompression = InvalidOid; diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c index 3ad4a61739..1682afd2a4 100644 --- a/src/backend/commands/amcmds.c +++ b/src/backend/commands/amcmds.c @@ -13,8 +13,11 @@ */ #include "postgres.h" +#include "access/amapi.h" +#include "access/compressamapi.h" #include "access/htup_details.h" #include "access/table.h" +#include "access/xact.h" #include "catalog/catalog.h" #include "catalog/dependency.h" #include "catalog/indexing.h" @@ -27,13 +30,20 @@ #include "parser/parse_func.h" #include "utils/builtins.h" #include "utils/lsyscache.h" +#include "utils/guc.h" +#include "utils/inval.h" #include "utils/rel.h" #include "utils/syscache.h" - static Oid lookup_am_handler_func(List *handler_name, char amtype); static const char *get_am_type_string(char amtype); +/* Compile-time default */ +char *default_toast_compression = DEFAULT_TOAST_COMPRESSION; // maybe needs pg_dump support ? +/* Invalid means need to lookup the text value in the catalog */ +static Oid default_toast_compression_oid = InvalidOid; + +static void AccessMethodCallback(Datum arg, int cacheid, uint32 hashvalue); /* * CreateAccessMethod @@ -277,3 +287,134 @@ lookup_am_handler_func(List *handler_name, char amtype) return handlerOid; } + +/* check_hook: validate new default_toast_compression */ +bool +check_default_toast_compression(char **newval, void **extra, GucSource source) +{ + if (**newval == '\0') + { + GUC_check_errdetail("%s cannot be empty.", + "default_toast_compression"); + return false; + } + + if (strlen(*newval) >= NAMEDATALEN) + { + GUC_check_errdetail("%s is too long (maximum %d characters).", + "default_toast_compression", NAMEDATALEN - 1); + return false; + } + + /* + * If we aren't inside a transaction, or not connected to a database, we + * cannot do the catalog access necessary to verify the method. Must + * accept the value on faith. + */ + if (IsTransactionState() && MyDatabaseId != InvalidOid) + { + if (!OidIsValid(get_compression_am_oid(*newval, true))) + { + /* + * When source == PGC_S_TEST, don't throw a hard error for a + * nonexistent table access method, only a NOTICE. See comments in + * guc.h. + */ + if (source == PGC_S_TEST) + { + ereport(NOTICE, + (errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("compression access method \"%s\" does not exist", + *newval))); + } + else + { + GUC_check_errdetail("Compression access method \"%s\" does not exist.", + *newval); + return false; + } + } + } + + return true; +} + +/* + * assign_default_toast_compression: GUC assign_hook for default_toast_compression + */ +void +assign_default_toast_compression(const char *newval, void *extra) +{ + /* + * Invalidate setting, forcing it to be looked up as needed. + * This avoids trying to do database access during GUC initialization, + * or outside a transaction. + */ + + default_toast_compression = NULL; +fprintf(stderr, "set to null\n"); +} + + +/* + * GetDefaultToastCompression -- get the OID of the current toast compression + * + * This exists to hide and optimize the use of the default_toast_compression + * GUC variable. + */ +Oid +GetDefaultToastCompression(void) +{ + /* Avoid catalog access during bootstrap, and for default compression */ + if (strcmp(default_toast_compression, DEFAULT_TOAST_COMPRESSION) == 0) + return PGLZ_COMPRESSION_AM_OID; + + /* Cannot call get_compression_am_oid this early */ + // if (IsBootstrapProcessingMode()) + // return PGLZ_COMPRESSION_AM_OID; + Assert(!IsBootstrapProcessingMode()); + + /* + * If cached value isn't valid, look up the current default value, caching + * the result + */ + if (!OidIsValid(default_toast_compression_oid)) + default_toast_compression_oid = + get_am_type_oid(default_toast_compression, AMTYPE_COMPRESSION, + false); + + return default_toast_compression_oid; +} + +/* + * InitializeAccessMethods: initialize module during InitPostgres. + * + * This is called after we are up enough to be able to do catalog lookups. + */ +void +InitializeAccessMethods(void) +{ + if (IsBootstrapProcessingMode()) + return; + + /* + * In normal mode, arrange for a callback on any syscache invalidation + * of pg_am rows. + */ + CacheRegisterSyscacheCallback(AMOID, + AccessMethodCallback, + (Datum) 0); + /* Force cached default access method to be recomputed on next use */ + // default_toast_compression_oid = InvalidOid; +} + +/* + * AccessMethodCallback + * Syscache inval callback function + */ +static void +AccessMethodCallback(Datum arg, int cacheid, uint32 hashvalue) +{ + /* Force look up of compression oid on next use */ + default_toast_compression_oid = false; +} diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index dd81d5bf4e..72ba017814 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -11925,7 +11925,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel, if (!IsStorageCompressible(tform->typstorage)) attTup->attcompression = InvalidOid; else if (!OidIsValid(attTup->attcompression)) - attTup->attcompression = DefaultCompressionOid; + attTup->attcompression = GetDefaultToastCompression(); } else attTup->attcompression = InvalidOid; @@ -17762,7 +17762,7 @@ GetAttributeCompression(Form_pg_attribute att, char *compression) /* fallback to default compression if it's not specified */ if (compression == NULL) - return DefaultCompressionOid; + return GetDefaultToastCompression(); amoid = get_compression_am_oid(compression, false); diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c index e5965bc517..6a02bbd377 100644 --- a/src/backend/utils/init/postinit.c +++ b/src/backend/utils/init/postinit.c @@ -25,6 +25,7 @@ #include "access/session.h" #include "access/sysattr.h" #include "access/tableam.h" +#include "access/amapi.h" #include "access/xact.h" #include "access/xlog.h" #include "catalog/catalog.h" @@ -1060,6 +1061,9 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username, /* set default namespace search path */ InitializeSearchPath(); + /* set callback for changes to pg_am */ + InitializeAccessMethods(); + /* initialize client encoding */ InitializeClientEncoding(); diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c index eafdb1118e..3a2b33fcd1 100644 --- a/src/backend/utils/misc/guc.c +++ b/src/backend/utils/misc/guc.c @@ -30,6 +30,7 @@ #include <unistd.h> #include "access/commit_ts.h" +#include "access/compressamapi.h" #include "access/gin.h" #include "access/rmgr.h" #include "access/tableam.h" @@ -3915,6 +3916,17 @@ static struct config_string ConfigureNamesString[] = check_default_table_access_method, NULL, NULL }, + { + {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT, + gettext_noop("Sets the default compression for new columns."), + NULL, + GUC_IS_NAME + }, + &default_toast_compression, + DEFAULT_TOAST_COMPRESSION, + check_default_toast_compression, assign_default_toast_compression, NULL, NULL + }, + { {"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT, gettext_noop("Sets the default tablespace to create tables and indexes in."), diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h index d357ebb559..1513cafcf4 100644 --- a/src/include/access/amapi.h +++ b/src/include/access/amapi.h @@ -287,4 +287,6 @@ typedef struct IndexAmRoutine extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler); extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror); +void InitializeAccessMethods(void); + #endif /* AMAPI_H */ diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h index 5a8e23d926..d75a8e9df2 100644 --- a/src/include/access/compressamapi.h +++ b/src/include/access/compressamapi.h @@ -17,6 +17,7 @@ #include "catalog/pg_am_d.h" #include "nodes/nodes.h" +#include "utils/guc.h" /* * Built-in compression method-id. The toast compression header will store @@ -29,8 +30,17 @@ typedef enum CompressionId LZ4_COMPRESSION_ID = 1 } CompressionId; -/* Use default compression method if it is not specified. */ +/* Default compression method if not specified. */ +#define DEFAULT_TOAST_COMPRESSION "pglz" #define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID + +/* GUC */ +extern char *default_toast_compression; +extern void assign_default_toast_compression(const char *newval, void *extra); +extern bool check_default_toast_compression(char **newval, void **extra, GucSource source); + +extern Oid GetDefaultToastCompression(void); + #define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \ (storage) != TYPSTORAGE_EXTERNAL) /* compression handler routines */ -- 2.17.0 --YZ5djTAD1cGYuMQK Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v24-0004-default-to-with-lz4.patch" ^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: OK to build LLVM (*.bc) with CLANG but rest of postgresql with CC (other compiler)? @ 2023-06-22 19:02 Andres Freund <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: Andres Freund @ 2023-06-22 19:02 UTC (permalink / raw) To: Palle Girgensohn <[email protected]>; +Cc: pgsql-hackers Hi, On 2023-06-13 11:20:52 +0200, Palle Girgensohn wrote: > CLANG is used to compile *.bc files during postgresql build. Is it OK to > have a different compiler for the rest of the build? gcc, or even another > version of clang? Yes. > LLVM is an optional add-on, a package. The default version is 15, and it also installs the clang15 binary for the corresponding clang version 15. > > As I understand, you're "better off" compiling the LLVM stuff in PostgreSQL with the same version clang compiler as the LLVM version you're using. Hence, with LLVM 15, set the environment variable CLANG=/path/to/clang15 when running configure. If the .bc files will get compiled by the base system clang compiler, this can lead to a ThinLTO link error, if the base system compiler is a newer versione of llvm. Yea, the compatibility matrix for llvm bitcode is complicated. > The question is if it is a bad idea to use the base compiler, say clang13, > to build postgresql, but set CLANG=clang15 to match the LLVM version. Am I > better off using clang15 for everything then? That should be entirely fine. If you already have the newer clang version, it might also make sense to just use it from a simplicity perspective (e.g. compiler warnings being the same etc), but it's not required. It works just fine to compile the postgres binary with gcc and use clang for the bitcode after all. Greetings, Andres Freund ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2023-06-22 19:02 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-01-30 03:37 [PATCH v24 3/7] Add default_toast_compression GUC Justin Pryzby <[email protected]> 2023-06-22 19:02 Re: OK to build LLVM (*.bc) with CLANG but rest of postgresql with CC (other compiler)? Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox