agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH 7/8] Improve sys/catcache performance. 6+ messages / 4 participants [nested] [flat]
* [PATCH 7/8] Improve sys/catcache performance. @ 2017-09-12 01:25 Andres Freund <[email protected]> 0 siblings, 0 replies; 6+ messages in thread From: Andres Freund @ 2017-09-12 01:25 UTC (permalink / raw) This primarily includes four pieces: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Only initializing the ScanKey when necessary, i.e. catcache misses, reduces cache unnecessary cpu cache misses. 3) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 4) Split of the heap lookup from the hash lookup, reducing stack allocations etc in the common case. There's further potential: - replace open coded hash with simplehash - the list walk right now shows up in profiles. - As oid is the only system column supported, avoid the use of heap_getsysattr(), by adding an explicit branch for ObjectIdAttributeNumber. This shows up in profiles. - move cache initialization out of the search path - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund --- src/backend/utils/cache/catcache.c | 438 ++++++++++++++++++++++++++----------- src/backend/utils/cache/syscache.c | 49 ++++- src/include/utils/catcache.h | 18 +- src/include/utils/syscache.h | 23 +- 4 files changed, 390 insertions(+), 138 deletions(-) diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index e092801025..67c596d29b 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,6 +31,7 @@ #include "storage/lmgr.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/hashutils.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" @@ -72,11 +73,25 @@ /* Cache management header --- pointer is NULL until created */ static CatCacheHeader *CacheHdr = NULL; +static inline HeapTuple SearchCatCacheInternal(CatCache *cache, + int nkeys, + Datum v1, Datum v2, + Datum v3, Datum v4); + +static pg_noinline HeapTuple SearchCatCacheMiss(CatCache *cache, + int nkeys, + uint32 hashValue, + Index hashIndex, + Datum v1, Datum v2, + Datum v3, Datum v4); static uint32 CatalogCacheComputeHashValue(CatCache *cache, int nkeys, - ScanKey cur_skey); -static uint32 CatalogCacheComputeTupleHashValue(CatCache *cache, + Datum v1, Datum v2, Datum v3, Datum v4); +static uint32 CatalogCacheComputeTupleHashValue(CatCache *cache, int nkeys, HeapTuple tuple); +static inline bool CatalogCacheCompareTuple(const CatCache *cache, int nkeys, + const HeapTuple tuple, + const Datum *arguments); #ifdef CATCACHE_STATS static void CatCachePrintStats(int code, Datum arg); @@ -95,45 +110,127 @@ static HeapTuple build_dummy_tuple(CatCache *cache, int nkeys, ScanKey skeys); */ /* - * Look up the hash and equality functions for system types that are used - * as cache key fields. - * - * XXX this should be replaced by catalog lookups, - * but that seems to pose considerable risk of circularity... + * Hash and equality functions for system types that are used as cache key + * fields. To compute hashes, and to check for hash collisions, use functions + * hardcoded for that purpose. This is sufficiently performance critical that + * the overhead of SQL style function calls is noticeable. */ + +static bool +chareqfast(Datum a, Datum b) +{ + return DatumGetChar(a) == DatumGetChar(b); +} + +static uint32 +charhashfast(Datum datum) +{ + return murmurhash32((int32) DatumGetChar(datum)); +} + +static bool +nameeqfast(Datum a, Datum b) +{ + char *ca = NameStr(*DatumGetName(a)); + char *cb = NameStr(*DatumGetName(b)); + + return strncmp(ca, cb, NAMEDATALEN) == 0; +} + +static uint32 +namehashfast(Datum datum) +{ + char *key = NameStr(*DatumGetName(datum)); + + return hash_any((unsigned char *) key, strlen(key)); +} + +static bool +int2eqfast(Datum a, Datum b) +{ + return DatumGetInt16(a) == DatumGetInt16(b); +} + +static uint32 +int2hashfast(Datum datum) +{ + return murmurhash32((int32) DatumGetInt16(datum)); +} + +static bool +int4eqfast(Datum a, Datum b) +{ + return DatumGetInt32(a) == DatumGetInt32(b); +} + +static uint32 +int4hashfast(Datum datum) +{ + return murmurhash32((int32) DatumGetInt32(datum)); +} + +static bool +texteqfast(Datum a, Datum b) +{ + /* not as performance critical & "complicated" */ + return DatumGetBool(DirectFunctionCall2(texteq, a, b)); +} + +static uint32 +texthashfast(Datum datum) +{ + /* not as performance critical & "complicated" */ + return DatumGetInt32(DirectFunctionCall1(hashtext, datum)); +} + +static bool +oidvectoreqfast(Datum a, Datum b) +{ + /* not as performance critical & "complicated" */ + return DatumGetBool(DirectFunctionCall2(oidvectoreq, a, b)); +} + +static uint32 +oidvectorhashfast(Datum datum) +{ + /* not as performance critical & "complicated" */ + return DatumGetInt32(DirectFunctionCall1(hashoidvector, datum)); +} + +/* Lookup support functions for a type. */ static void -GetCCHashEqFuncs(Oid keytype, PGFunction *hashfunc, RegProcedure *eqfunc) +GetCCHashEqFuncs(Oid keytype, CCHashFN *hashfunc, RegProcedure *eqfunc, CCFastEqualFN *fasteqfunc) { switch (keytype) { case BOOLOID: - *hashfunc = hashchar; - + *hashfunc = charhashfast; + *fasteqfunc = chareqfast; *eqfunc = F_BOOLEQ; break; case CHAROID: - *hashfunc = hashchar; - + *hashfunc = charhashfast; + *fasteqfunc = chareqfast; *eqfunc = F_CHAREQ; break; case NAMEOID: - *hashfunc = hashname; - + *hashfunc = namehashfast; + *fasteqfunc = nameeqfast; *eqfunc = F_NAMEEQ; break; case INT2OID: - *hashfunc = hashint2; - + *hashfunc = int2hashfast; + *fasteqfunc = int2eqfast; *eqfunc = F_INT2EQ; break; case INT4OID: - *hashfunc = hashint4; - + *hashfunc = int4hashfast; + *fasteqfunc = int4eqfast; *eqfunc = F_INT4EQ; break; case TEXTOID: - *hashfunc = hashtext; - + *hashfunc = texthashfast; + *fasteqfunc = texteqfast; *eqfunc = F_TEXTEQ; break; case OIDOID: @@ -147,13 +244,13 @@ GetCCHashEqFuncs(Oid keytype, PGFunction *hashfunc, RegProcedure *eqfunc) case REGDICTIONARYOID: case REGROLEOID: case REGNAMESPACEOID: - *hashfunc = hashoid; - + *hashfunc = int4hashfast; + *fasteqfunc = int4eqfast; *eqfunc = F_OIDEQ; break; case OIDVECTOROID: - *hashfunc = hashoidvector; - + *hashfunc = oidvectorhashfast; + *fasteqfunc = oidvectoreqfast; *eqfunc = F_OIDVECTOREQ; break; default: @@ -171,10 +268,12 @@ GetCCHashEqFuncs(Oid keytype, PGFunction *hashfunc, RegProcedure *eqfunc) * Compute the hash value associated with a given set of lookup keys */ static uint32 -CatalogCacheComputeHashValue(CatCache *cache, int nkeys, ScanKey cur_skey) +CatalogCacheComputeHashValue(CatCache *cache, int nkeys, + Datum v1, Datum v2, Datum v3, Datum v4) { uint32 hashValue = 0; uint32 oneHash; + CCHashFN *cc_hashfunc = cache->cc_hashfunc; CACHE4_elog(DEBUG2, "CatalogCacheComputeHashValue %s %d %p", cache->cc_relname, @@ -184,30 +283,26 @@ CatalogCacheComputeHashValue(CatCache *cache, int nkeys, ScanKey cur_skey) switch (nkeys) { case 4: - oneHash = - DatumGetUInt32(DirectFunctionCall1(cache->cc_hashfunc[3], - cur_skey[3].sk_argument)); + oneHash = (cc_hashfunc[3])(v4); + hashValue ^= oneHash << 24; hashValue ^= oneHash >> 8; /* FALLTHROUGH */ case 3: - oneHash = - DatumGetUInt32(DirectFunctionCall1(cache->cc_hashfunc[2], - cur_skey[2].sk_argument)); + oneHash = (cc_hashfunc[2])(v3); + hashValue ^= oneHash << 16; hashValue ^= oneHash >> 16; /* FALLTHROUGH */ case 2: - oneHash = - DatumGetUInt32(DirectFunctionCall1(cache->cc_hashfunc[1], - cur_skey[1].sk_argument)); + oneHash = (cc_hashfunc[1])(v2); + hashValue ^= oneHash << 8; hashValue ^= oneHash >> 24; /* FALLTHROUGH */ case 1: - oneHash = - DatumGetUInt32(DirectFunctionCall1(cache->cc_hashfunc[0], - cur_skey[0].sk_argument)); + oneHash = (cc_hashfunc[0])(v1); + hashValue ^= oneHash; break; default: @@ -224,63 +319,96 @@ CatalogCacheComputeHashValue(CatCache *cache, int nkeys, ScanKey cur_skey) * Compute the hash value associated with a given tuple to be cached */ static uint32 -CatalogCacheComputeTupleHashValue(CatCache *cache, HeapTuple tuple) +CatalogCacheComputeTupleHashValue(CatCache *cache, int nkeys, HeapTuple tuple) { - ScanKeyData cur_skey[CATCACHE_MAXKEYS]; + Datum v1 = 0, v2 = 0, v3 = 0, v4 = 0; bool isNull = false; - - /* Copy pre-initialized overhead data for scankey */ - memcpy(cur_skey, cache->cc_skey, sizeof(cur_skey)); + int *cc_key = cache->cc_key; + TupleDesc cc_tupdesc = cache->cc_tupdesc; /* Now extract key fields from tuple, insert into scankey */ - switch (cache->cc_nkeys) + switch (nkeys) { case 4: - cur_skey[3].sk_argument = - (cache->cc_key[3] == ObjectIdAttributeNumber) + v4 = (cc_key[3] == ObjectIdAttributeNumber) ? ObjectIdGetDatum(HeapTupleGetOid(tuple)) : fastgetattr(tuple, - cache->cc_key[3], - cache->cc_tupdesc, + cc_key[3], + cc_tupdesc, &isNull); Assert(!isNull); /* FALLTHROUGH */ case 3: - cur_skey[2].sk_argument = - (cache->cc_key[2] == ObjectIdAttributeNumber) + v3 = (cc_key[2] == ObjectIdAttributeNumber) ? ObjectIdGetDatum(HeapTupleGetOid(tuple)) : fastgetattr(tuple, - cache->cc_key[2], - cache->cc_tupdesc, + cc_key[2], + cc_tupdesc, &isNull); Assert(!isNull); /* FALLTHROUGH */ case 2: - cur_skey[1].sk_argument = - (cache->cc_key[1] == ObjectIdAttributeNumber) + v2 = (cc_key[1] == ObjectIdAttributeNumber) ? ObjectIdGetDatum(HeapTupleGetOid(tuple)) : fastgetattr(tuple, - cache->cc_key[1], - cache->cc_tupdesc, + cc_key[1], + cc_tupdesc, &isNull); Assert(!isNull); /* FALLTHROUGH */ case 1: - cur_skey[0].sk_argument = - (cache->cc_key[0] == ObjectIdAttributeNumber) + v1 = (cc_key[0] == ObjectIdAttributeNumber) ? ObjectIdGetDatum(HeapTupleGetOid(tuple)) : fastgetattr(tuple, - cache->cc_key[0], - cache->cc_tupdesc, + cc_key[0], + cc_tupdesc, &isNull); Assert(!isNull); break; default: - elog(FATAL, "wrong number of hash keys: %d", cache->cc_nkeys); + elog(FATAL, "wrong number of hash keys: %d", nkeys); break; } - return CatalogCacheComputeHashValue(cache, cache->cc_nkeys, cur_skey); + return CatalogCacheComputeHashValue(cache, nkeys, v1, v2, v3, v4); +} + +/* + * CatalogCacheCompareTuple + * + * Compare a tuple to the passed arguments. + */ +static inline bool +CatalogCacheCompareTuple(const CatCache *cache, int nkeys, + const HeapTuple tuple, + const Datum *arguments) +{ + TupleDesc tupdesc = cache->cc_tupdesc; + const int *cc_key = cache->cc_key; + const CCFastEqualFN *cc_fastequal = cache->cc_fastequal; + int i; + + for (i = 0; i < nkeys; i++) + { + Datum atp; + bool isnull; + + /* + * XXX: might be worthwhile to only handle oid sysattr, to reduce + * overhead - it's the most common key. + */ + atp = heap_getattr(tuple, + cc_key[i], + tupdesc, + &isnull); + Assert(!isnull); + + if (!(cc_fastequal[i])(atp, arguments[i])) + { + return false; + } + } + return true; } @@ -878,7 +1006,8 @@ CatalogCacheInitializeCache(CatCache *cache) GetCCHashEqFuncs(keytype, &cache->cc_hashfunc[i], - &eqfunc); + &eqfunc, + &cache->cc_fastequal[i]); cache->cc_isname[i] = (keytype == NAMEOID); @@ -1020,7 +1149,7 @@ IndexScanOK(CatCache *cache, ScanKey cur_skey) } /* - * SearchCatCache + * SearchCatCacheInternal * * This call searches a system cache for a tuple, opening the relation * if necessary (on the first access to a particular cache). @@ -1042,15 +1171,64 @@ SearchCatCache(CatCache *cache, Datum v3, Datum v4) { - ScanKeyData cur_skey[CATCACHE_MAXKEYS]; + return SearchCatCacheInternal(cache, cache->cc_nkeys, v1, v2, v3, v4); +} + + +/* + * SearchCatCacheN() are SearchCatCache() versions for a specific number of + * arguments. The compiler can inline the body and unroll the loop, making + * them a bit faster than SearchCatCache(). + */ + +HeapTuple +SearchCatCache1(CatCache *cache, + Datum v1) +{ + return SearchCatCacheInternal(cache, 1, v1, 0, 0, 0); +} + + +HeapTuple +SearchCatCache2(CatCache *cache, + Datum v1, Datum v2) +{ + return SearchCatCacheInternal(cache, 2, v1, v2, 0, 0); +} + + +HeapTuple +SearchCatCache3(CatCache *cache, + Datum v1, Datum v2, Datum v3) +{ + return SearchCatCacheInternal(cache, 3, v1, v2, v3, 0); +} + + +HeapTuple +SearchCatCache4(CatCache *cache, + Datum v1, Datum v2, Datum v3, Datum v4) +{ + return SearchCatCacheInternal(cache, 4, v1, v2, v3, v4); +} + +/* + * Work-horse for SearchCatCache/SearchCatCacheN. + */ +static inline HeapTuple +SearchCatCacheInternal(CatCache *cache, + int nkeys, + Datum v1, + Datum v2, + Datum v3, + Datum v4) +{ + Datum arguments[CATCACHE_MAXKEYS]; uint32 hashValue; Index hashIndex; dlist_iter iter; dlist_head *bucket; CatCTup *ct; - Relation relation; - SysScanDesc scandesc; - HeapTuple ntp; /* Make sure we're in an xact, even if this ends up being a cache hit */ Assert(IsTransactionState()); @@ -1058,26 +1236,23 @@ SearchCatCache(CatCache *cache, /* * one-time startup overhead for each cache */ - if (cache->cc_tupdesc == NULL) + if (unlikely(cache->cc_tupdesc == NULL)) CatalogCacheInitializeCache(cache); #ifdef CATCACHE_STATS cache->cc_searches++; #endif - /* - * initialize the search key information - */ - memcpy(cur_skey, cache->cc_skey, sizeof(cur_skey)); - cur_skey[0].sk_argument = v1; - cur_skey[1].sk_argument = v2; - cur_skey[2].sk_argument = v3; - cur_skey[3].sk_argument = v4; + /* Initialize local parameter array */ + arguments[0] = v1; + arguments[1] = v2; + arguments[2] = v3; + arguments[3] = v4; /* * find the hash bucket in which to look for the tuple */ - hashValue = CatalogCacheComputeHashValue(cache, cache->cc_nkeys, cur_skey); + hashValue = CatalogCacheComputeHashValue(cache, nkeys, v1, v2, v3, v4); hashIndex = HASH_INDEX(hashValue, cache->cc_nbuckets); /* @@ -1089,8 +1264,6 @@ SearchCatCache(CatCache *cache, bucket = &cache->cc_bucket[hashIndex]; dlist_foreach(iter, bucket) { - bool res; - ct = dlist_container(CatCTup, cache_elem, iter.cur); if (ct->dead) @@ -1099,15 +1272,7 @@ SearchCatCache(CatCache *cache, if (ct->hash_value != hashValue) continue; /* quickly skip entry if wrong hash val */ - /* - * see if the cached tuple matches our key. - */ - HeapKeyTest(&ct->tuple, - cache->cc_tupdesc, - cache->cc_nkeys, - cur_skey, - res); - if (!res) + if (!CatalogCacheCompareTuple(cache, nkeys, &ct->tuple, arguments)) continue; /* @@ -1150,6 +1315,42 @@ SearchCatCache(CatCache *cache, } } + return SearchCatCacheMiss(cache, nkeys, hashValue, hashIndex, v1, v2, v3, v4); +} + +/* + * Search the actual catalogs, rather than the cache. + * + * This is kept separate from SearchCatCacheInternal() to keep the fast-path + * as small as possible. To avoid that effort being undone, try to explicitly + * forbid inlining. + */ +static pg_noinline HeapTuple +SearchCatCacheMiss(CatCache *cache, + int nkeys, + uint32 hashValue, + Index hashIndex, + Datum v1, + Datum v2, + Datum v3, + Datum v4) +{ + ScanKeyData cur_skey[CATCACHE_MAXKEYS]; + Relation relation; + SysScanDesc scandesc; + HeapTuple ntp; + CatCTup *ct; + + /* + * Ok, need to make a lookup in the relation, copy the scankey and fill out + * any per-call fields. + */ + memcpy(cur_skey, cache->cc_skey, sizeof(ScanKeyData) * nkeys); + cur_skey[0].sk_argument = v1; + cur_skey[1].sk_argument = v2; + cur_skey[2].sk_argument = v3; + cur_skey[3].sk_argument = v4; + /* * Tuple was not found in cache, so we have to try to retrieve it directly * from the relation. If found, we will add it to the cache; if not @@ -1171,7 +1372,7 @@ SearchCatCache(CatCache *cache, cache->cc_indexoid, IndexScanOK(cache, cur_skey), NULL, - cache->cc_nkeys, + nkeys, cur_skey); ct = NULL; @@ -1207,7 +1408,7 @@ SearchCatCache(CatCache *cache, if (IsBootstrapProcessingMode()) return NULL; - ntp = build_dummy_tuple(cache, cache->cc_nkeys, cur_skey); + ntp = build_dummy_tuple(cache, nkeys, cur_skey); ct = CatalogCacheCreateEntry(cache, ntp, hashValue, hashIndex, true); @@ -1288,27 +1489,16 @@ GetCatCacheHashValue(CatCache *cache, Datum v3, Datum v4) { - ScanKeyData cur_skey[CATCACHE_MAXKEYS]; - /* * one-time startup overhead for each cache */ if (cache->cc_tupdesc == NULL) CatalogCacheInitializeCache(cache); - /* - * initialize the search key information - */ - memcpy(cur_skey, cache->cc_skey, sizeof(cur_skey)); - cur_skey[0].sk_argument = v1; - cur_skey[1].sk_argument = v2; - cur_skey[2].sk_argument = v3; - cur_skey[3].sk_argument = v4; - /* * calculate the hash value */ - return CatalogCacheComputeHashValue(cache, cache->cc_nkeys, cur_skey); + return CatalogCacheComputeHashValue(cache, cache->cc_nkeys, v1, v2, v3, v4); } @@ -1329,7 +1519,7 @@ SearchCatCacheList(CatCache *cache, Datum v3, Datum v4) { - ScanKeyData cur_skey[CATCACHE_MAXKEYS]; + Datum arguments[CATCACHE_MAXKEYS]; uint32 lHashValue; dlist_iter iter; CatCList *cl; @@ -1354,21 +1544,18 @@ SearchCatCacheList(CatCache *cache, cache->cc_lsearches++; #endif - /* - * initialize the search key information - */ - memcpy(cur_skey, cache->cc_skey, sizeof(cur_skey)); - cur_skey[0].sk_argument = v1; - cur_skey[1].sk_argument = v2; - cur_skey[2].sk_argument = v3; - cur_skey[3].sk_argument = v4; + /* Initialize local parameter array */ + arguments[0] = v1; + arguments[1] = v2; + arguments[2] = v3; + arguments[3] = v4; /* * compute a hash value of the given keys for faster search. We don't * presently divide the CatCList items into buckets, but this still lets * us skip non-matching items quickly most of the time. */ - lHashValue = CatalogCacheComputeHashValue(cache, nkeys, cur_skey); + lHashValue = CatalogCacheComputeHashValue(cache, nkeys, v1, v2, v3, v4); /* * scan the items until we find a match or exhaust our list @@ -1378,8 +1565,6 @@ SearchCatCacheList(CatCache *cache, */ dlist_foreach(iter, &cache->cc_lists) { - bool res; - cl = dlist_container(CatCList, cache_elem, iter.cur); if (cl->dead) @@ -1393,12 +1578,8 @@ SearchCatCacheList(CatCache *cache, */ if (cl->nkeys != nkeys) continue; - HeapKeyTest(&cl->tuple, - cache->cc_tupdesc, - nkeys, - cur_skey, - res); - if (!res) + + if (!CatalogCacheCompareTuple(cache, nkeys, &cl->tuple, arguments)) continue; /* @@ -1441,9 +1622,20 @@ SearchCatCacheList(CatCache *cache, PG_TRY(); { + ScanKeyData cur_skey[CATCACHE_MAXKEYS]; Relation relation; SysScanDesc scandesc; + /* + * Ok, need to make a lookup in the relation, copy the scankey and fill out + * any per-call fields. + */ + memcpy(cur_skey, cache->cc_skey, sizeof(ScanKeyData) * cache->cc_nkeys); + cur_skey[0].sk_argument = v1; + cur_skey[1].sk_argument = v2; + cur_skey[2].sk_argument = v3; + cur_skey[3].sk_argument = v4; + relation = heap_open(cache->cc_reloid, AccessShareLock); scandesc = systable_beginscan(relation, @@ -1467,7 +1659,7 @@ SearchCatCacheList(CatCache *cache, * See if there's an entry for this tuple already. */ ct = NULL; - hashValue = CatalogCacheComputeTupleHashValue(cache, ntp); + hashValue = CatalogCacheComputeTupleHashValue(cache, cache->cc_nkeys, ntp); hashIndex = HASH_INDEX(hashValue, cache->cc_nbuckets); bucket = &cache->cc_bucket[hashIndex]; @@ -1820,7 +2012,7 @@ PrepareToInvalidateCacheTuple(Relation relation, if (ccp->cc_tupdesc == NULL) CatalogCacheInitializeCache(ccp); - hashvalue = CatalogCacheComputeTupleHashValue(ccp, tuple); + hashvalue = CatalogCacheComputeTupleHashValue(ccp, ccp->cc_nkeys, tuple); dbid = ccp->cc_relisshared ? (Oid) 0 : MyDatabaseId; (*function) (ccp->id, hashvalue, dbid); @@ -1829,7 +2021,7 @@ PrepareToInvalidateCacheTuple(Relation relation, { uint32 newhashvalue; - newhashvalue = CatalogCacheComputeTupleHashValue(ccp, newtuple); + newhashvalue = CatalogCacheComputeTupleHashValue(ccp, ccp->cc_nkeys, newtuple); if (newhashvalue != hashvalue) (*function) (ccp->id, newhashvalue, dbid); diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index fcbb683a99..888edbb325 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -1102,13 +1102,56 @@ SearchSysCache(int cacheId, Datum key3, Datum key4) { - if (cacheId < 0 || cacheId >= SysCacheSize || - !PointerIsValid(SysCache[cacheId])) - elog(ERROR, "invalid cache ID: %d", cacheId); + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); return SearchCatCache(SysCache[cacheId], key1, key2, key3, key4); } +HeapTuple +SearchSysCache1(int cacheId, + Datum key1) +{ + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); + Assert(SysCache[cacheId]->cc_nkeys == 1); + + return SearchCatCache1(SysCache[cacheId], key1); +} + +HeapTuple +SearchSysCache2(int cacheId, + Datum key1, Datum key2) +{ + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); + Assert(SysCache[cacheId]->cc_nkeys == 2); + + return SearchCatCache2(SysCache[cacheId], key1, key2); +} + +HeapTuple +SearchSysCache3(int cacheId, + Datum key1, Datum key2, Datum key3) +{ + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); + Assert(SysCache[cacheId]->cc_nkeys == 3); + + return SearchCatCache3(SysCache[cacheId], key1, key2, key3); +} + +HeapTuple +SearchSysCache4(int cacheId, + Datum key1, Datum key2, Datum key3, Datum key4) +{ + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); + Assert(SysCache[cacheId]->cc_nkeys == 4); + + return SearchCatCache4(SysCache[cacheId], key1, key2, key3, key4); +} + /* * ReleaseSysCache * Release previously grabbed reference count on a tuple diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index 200a3022e7..360f0c5dd5 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -34,6 +34,10 @@ #define CATCACHE_MAXKEYS 4 + +typedef uint32 (*CCHashFN) (Datum datum); +typedef bool (*CCFastEqualFN) (Datum a, Datum b); + typedef struct catcache { int id; /* cache identifier --- see syscache.h */ @@ -47,7 +51,8 @@ typedef struct catcache int cc_nbuckets; /* # of hash buckets in this cache */ int cc_nkeys; /* # of keys (1..CATCACHE_MAXKEYS) */ int cc_key[CATCACHE_MAXKEYS]; /* AttrNumber of each key */ - PGFunction cc_hashfunc[CATCACHE_MAXKEYS]; /* hash function for each key */ + CCHashFN cc_hashfunc[CATCACHE_MAXKEYS]; /* hash function for each key */ + CCFastEqualFN cc_fastequal[CATCACHE_MAXKEYS]; /* fast equal function for each key */ ScanKeyData cc_skey[CATCACHE_MAXKEYS]; /* precomputed key info for heap * scans */ bool cc_isname[CATCACHE_MAXKEYS]; /* flag "name" key columns */ @@ -174,8 +179,15 @@ extern CatCache *InitCatCache(int id, Oid reloid, Oid indexoid, extern void InitCatCachePhase2(CatCache *cache, bool touch_index); extern HeapTuple SearchCatCache(CatCache *cache, - Datum v1, Datum v2, - Datum v3, Datum v4); + Datum v1, Datum v2, Datum v3, Datum v4); +extern HeapTuple SearchCatCache1(CatCache *cache, + Datum v1); +extern HeapTuple SearchCatCache2(CatCache *cache, + Datum v1, Datum v2); +extern HeapTuple SearchCatCache3(CatCache *cache, + Datum v1, Datum v2, Datum v3); +extern HeapTuple SearchCatCache4(CatCache *cache, + Datum v1, Datum v2, Datum v3, Datum v4); extern void ReleaseCatCache(HeapTuple tuple); extern uint32 GetCatCacheHashValue(CatCache *cache, diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h index 8a92ea27ac..12bda02cd7 100644 --- a/src/include/utils/syscache.h +++ b/src/include/utils/syscache.h @@ -117,6 +117,20 @@ extern void InitCatalogCachePhase2(void); extern HeapTuple SearchSysCache(int cacheId, Datum key1, Datum key2, Datum key3, Datum key4); + +/* + * The use of argument specific numbers is encouraged, they're faster, and + * insulates the caller from changes in the maximum number of keys. + */ +extern HeapTuple SearchSysCache1(int cacheId, + Datum key1); +extern HeapTuple SearchSysCache2(int cacheId, + Datum key1, Datum key2); +extern HeapTuple SearchSysCache3(int cacheId, + Datum key1, Datum key2, Datum key3); +extern HeapTuple SearchSysCache4(int cacheId, + Datum key1, Datum key2, Datum key3, Datum key4); + extern void ReleaseSysCache(HeapTuple tuple); /* convenience routines */ @@ -156,15 +170,6 @@ extern bool RelationSupportsSysCache(Oid relid); * functions is encouraged, as it insulates the caller from changes in the * maximum number of keys. */ -#define SearchSysCache1(cacheId, key1) \ - SearchSysCache(cacheId, key1, 0, 0, 0) -#define SearchSysCache2(cacheId, key1, key2) \ - SearchSysCache(cacheId, key1, key2, 0, 0) -#define SearchSysCache3(cacheId, key1, key2, key3) \ - SearchSysCache(cacheId, key1, key2, key3, 0) -#define SearchSysCache4(cacheId, key1, key2, key3, key4) \ - SearchSysCache(cacheId, key1, key2, key3, key4) - #define SearchSysCacheCopy1(cacheId, key1) \ SearchSysCacheCopy(cacheId, key1, 0, 0, 0) #define SearchSysCacheCopy2(cacheId, key1, key2) \ -- 2.14.1.536.g6867272d5b.dirty --2rvhjmunxfshmpn5 Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 8bit MIME-Version: 1.0 -- Sent via pgsql-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers --2rvhjmunxfshmpn5-- ^ permalink raw reply [nested|flat] 6+ messages in thread
* [PATCH 7/8] Improve sys/catcache performance. @ 2017-09-12 01:25 Andres Freund <[email protected]> 0 siblings, 0 replies; 6+ messages in thread From: Andres Freund @ 2017-09-12 01:25 UTC (permalink / raw) This primarily includes four pieces: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Only initializing the ScanKey when necessary, i.e. catcache misses, reduces cache unnecessary cpu cache misses. 3) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 4) Split of the heap lookup from the hash lookup, reducing stack allocations etc in the common case. There's further potential: - replace open coded hash with simplehash - the list walk right now shows up in profiles. - As oid is the only system column supported, avoid the use of heap_getsysattr(), by adding an explicit branch for ObjectIdAttributeNumber. This shows up in profiles. - move cache initialization out of the search path - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund --- src/backend/utils/cache/catcache.c | 438 ++++++++++++++++++++++++++----------- src/backend/utils/cache/syscache.c | 49 ++++- src/include/utils/catcache.h | 18 +- src/include/utils/syscache.h | 23 +- 4 files changed, 390 insertions(+), 138 deletions(-) diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index e092801025..67c596d29b 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -31,6 +31,7 @@ #include "storage/lmgr.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/hashutils.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" @@ -72,11 +73,25 @@ /* Cache management header --- pointer is NULL until created */ static CatCacheHeader *CacheHdr = NULL; +static inline HeapTuple SearchCatCacheInternal(CatCache *cache, + int nkeys, + Datum v1, Datum v2, + Datum v3, Datum v4); + +static pg_noinline HeapTuple SearchCatCacheMiss(CatCache *cache, + int nkeys, + uint32 hashValue, + Index hashIndex, + Datum v1, Datum v2, + Datum v3, Datum v4); static uint32 CatalogCacheComputeHashValue(CatCache *cache, int nkeys, - ScanKey cur_skey); -static uint32 CatalogCacheComputeTupleHashValue(CatCache *cache, + Datum v1, Datum v2, Datum v3, Datum v4); +static uint32 CatalogCacheComputeTupleHashValue(CatCache *cache, int nkeys, HeapTuple tuple); +static inline bool CatalogCacheCompareTuple(const CatCache *cache, int nkeys, + const HeapTuple tuple, + const Datum *arguments); #ifdef CATCACHE_STATS static void CatCachePrintStats(int code, Datum arg); @@ -95,45 +110,127 @@ static HeapTuple build_dummy_tuple(CatCache *cache, int nkeys, ScanKey skeys); */ /* - * Look up the hash and equality functions for system types that are used - * as cache key fields. - * - * XXX this should be replaced by catalog lookups, - * but that seems to pose considerable risk of circularity... + * Hash and equality functions for system types that are used as cache key + * fields. To compute hashes, and to check for hash collisions, use functions + * hardcoded for that purpose. This is sufficiently performance critical that + * the overhead of SQL style function calls is noticeable. */ + +static bool +chareqfast(Datum a, Datum b) +{ + return DatumGetChar(a) == DatumGetChar(b); +} + +static uint32 +charhashfast(Datum datum) +{ + return murmurhash32((int32) DatumGetChar(datum)); +} + +static bool +nameeqfast(Datum a, Datum b) +{ + char *ca = NameStr(*DatumGetName(a)); + char *cb = NameStr(*DatumGetName(b)); + + return strncmp(ca, cb, NAMEDATALEN) == 0; +} + +static uint32 +namehashfast(Datum datum) +{ + char *key = NameStr(*DatumGetName(datum)); + + return hash_any((unsigned char *) key, strlen(key)); +} + +static bool +int2eqfast(Datum a, Datum b) +{ + return DatumGetInt16(a) == DatumGetInt16(b); +} + +static uint32 +int2hashfast(Datum datum) +{ + return murmurhash32((int32) DatumGetInt16(datum)); +} + +static bool +int4eqfast(Datum a, Datum b) +{ + return DatumGetInt32(a) == DatumGetInt32(b); +} + +static uint32 +int4hashfast(Datum datum) +{ + return murmurhash32((int32) DatumGetInt32(datum)); +} + +static bool +texteqfast(Datum a, Datum b) +{ + /* not as performance critical & "complicated" */ + return DatumGetBool(DirectFunctionCall2(texteq, a, b)); +} + +static uint32 +texthashfast(Datum datum) +{ + /* not as performance critical & "complicated" */ + return DatumGetInt32(DirectFunctionCall1(hashtext, datum)); +} + +static bool +oidvectoreqfast(Datum a, Datum b) +{ + /* not as performance critical & "complicated" */ + return DatumGetBool(DirectFunctionCall2(oidvectoreq, a, b)); +} + +static uint32 +oidvectorhashfast(Datum datum) +{ + /* not as performance critical & "complicated" */ + return DatumGetInt32(DirectFunctionCall1(hashoidvector, datum)); +} + +/* Lookup support functions for a type. */ static void -GetCCHashEqFuncs(Oid keytype, PGFunction *hashfunc, RegProcedure *eqfunc) +GetCCHashEqFuncs(Oid keytype, CCHashFN *hashfunc, RegProcedure *eqfunc, CCFastEqualFN *fasteqfunc) { switch (keytype) { case BOOLOID: - *hashfunc = hashchar; - + *hashfunc = charhashfast; + *fasteqfunc = chareqfast; *eqfunc = F_BOOLEQ; break; case CHAROID: - *hashfunc = hashchar; - + *hashfunc = charhashfast; + *fasteqfunc = chareqfast; *eqfunc = F_CHAREQ; break; case NAMEOID: - *hashfunc = hashname; - + *hashfunc = namehashfast; + *fasteqfunc = nameeqfast; *eqfunc = F_NAMEEQ; break; case INT2OID: - *hashfunc = hashint2; - + *hashfunc = int2hashfast; + *fasteqfunc = int2eqfast; *eqfunc = F_INT2EQ; break; case INT4OID: - *hashfunc = hashint4; - + *hashfunc = int4hashfast; + *fasteqfunc = int4eqfast; *eqfunc = F_INT4EQ; break; case TEXTOID: - *hashfunc = hashtext; - + *hashfunc = texthashfast; + *fasteqfunc = texteqfast; *eqfunc = F_TEXTEQ; break; case OIDOID: @@ -147,13 +244,13 @@ GetCCHashEqFuncs(Oid keytype, PGFunction *hashfunc, RegProcedure *eqfunc) case REGDICTIONARYOID: case REGROLEOID: case REGNAMESPACEOID: - *hashfunc = hashoid; - + *hashfunc = int4hashfast; + *fasteqfunc = int4eqfast; *eqfunc = F_OIDEQ; break; case OIDVECTOROID: - *hashfunc = hashoidvector; - + *hashfunc = oidvectorhashfast; + *fasteqfunc = oidvectoreqfast; *eqfunc = F_OIDVECTOREQ; break; default: @@ -171,10 +268,12 @@ GetCCHashEqFuncs(Oid keytype, PGFunction *hashfunc, RegProcedure *eqfunc) * Compute the hash value associated with a given set of lookup keys */ static uint32 -CatalogCacheComputeHashValue(CatCache *cache, int nkeys, ScanKey cur_skey) +CatalogCacheComputeHashValue(CatCache *cache, int nkeys, + Datum v1, Datum v2, Datum v3, Datum v4) { uint32 hashValue = 0; uint32 oneHash; + CCHashFN *cc_hashfunc = cache->cc_hashfunc; CACHE4_elog(DEBUG2, "CatalogCacheComputeHashValue %s %d %p", cache->cc_relname, @@ -184,30 +283,26 @@ CatalogCacheComputeHashValue(CatCache *cache, int nkeys, ScanKey cur_skey) switch (nkeys) { case 4: - oneHash = - DatumGetUInt32(DirectFunctionCall1(cache->cc_hashfunc[3], - cur_skey[3].sk_argument)); + oneHash = (cc_hashfunc[3])(v4); + hashValue ^= oneHash << 24; hashValue ^= oneHash >> 8; /* FALLTHROUGH */ case 3: - oneHash = - DatumGetUInt32(DirectFunctionCall1(cache->cc_hashfunc[2], - cur_skey[2].sk_argument)); + oneHash = (cc_hashfunc[2])(v3); + hashValue ^= oneHash << 16; hashValue ^= oneHash >> 16; /* FALLTHROUGH */ case 2: - oneHash = - DatumGetUInt32(DirectFunctionCall1(cache->cc_hashfunc[1], - cur_skey[1].sk_argument)); + oneHash = (cc_hashfunc[1])(v2); + hashValue ^= oneHash << 8; hashValue ^= oneHash >> 24; /* FALLTHROUGH */ case 1: - oneHash = - DatumGetUInt32(DirectFunctionCall1(cache->cc_hashfunc[0], - cur_skey[0].sk_argument)); + oneHash = (cc_hashfunc[0])(v1); + hashValue ^= oneHash; break; default: @@ -224,63 +319,96 @@ CatalogCacheComputeHashValue(CatCache *cache, int nkeys, ScanKey cur_skey) * Compute the hash value associated with a given tuple to be cached */ static uint32 -CatalogCacheComputeTupleHashValue(CatCache *cache, HeapTuple tuple) +CatalogCacheComputeTupleHashValue(CatCache *cache, int nkeys, HeapTuple tuple) { - ScanKeyData cur_skey[CATCACHE_MAXKEYS]; + Datum v1 = 0, v2 = 0, v3 = 0, v4 = 0; bool isNull = false; - - /* Copy pre-initialized overhead data for scankey */ - memcpy(cur_skey, cache->cc_skey, sizeof(cur_skey)); + int *cc_key = cache->cc_key; + TupleDesc cc_tupdesc = cache->cc_tupdesc; /* Now extract key fields from tuple, insert into scankey */ - switch (cache->cc_nkeys) + switch (nkeys) { case 4: - cur_skey[3].sk_argument = - (cache->cc_key[3] == ObjectIdAttributeNumber) + v4 = (cc_key[3] == ObjectIdAttributeNumber) ? ObjectIdGetDatum(HeapTupleGetOid(tuple)) : fastgetattr(tuple, - cache->cc_key[3], - cache->cc_tupdesc, + cc_key[3], + cc_tupdesc, &isNull); Assert(!isNull); /* FALLTHROUGH */ case 3: - cur_skey[2].sk_argument = - (cache->cc_key[2] == ObjectIdAttributeNumber) + v3 = (cc_key[2] == ObjectIdAttributeNumber) ? ObjectIdGetDatum(HeapTupleGetOid(tuple)) : fastgetattr(tuple, - cache->cc_key[2], - cache->cc_tupdesc, + cc_key[2], + cc_tupdesc, &isNull); Assert(!isNull); /* FALLTHROUGH */ case 2: - cur_skey[1].sk_argument = - (cache->cc_key[1] == ObjectIdAttributeNumber) + v2 = (cc_key[1] == ObjectIdAttributeNumber) ? ObjectIdGetDatum(HeapTupleGetOid(tuple)) : fastgetattr(tuple, - cache->cc_key[1], - cache->cc_tupdesc, + cc_key[1], + cc_tupdesc, &isNull); Assert(!isNull); /* FALLTHROUGH */ case 1: - cur_skey[0].sk_argument = - (cache->cc_key[0] == ObjectIdAttributeNumber) + v1 = (cc_key[0] == ObjectIdAttributeNumber) ? ObjectIdGetDatum(HeapTupleGetOid(tuple)) : fastgetattr(tuple, - cache->cc_key[0], - cache->cc_tupdesc, + cc_key[0], + cc_tupdesc, &isNull); Assert(!isNull); break; default: - elog(FATAL, "wrong number of hash keys: %d", cache->cc_nkeys); + elog(FATAL, "wrong number of hash keys: %d", nkeys); break; } - return CatalogCacheComputeHashValue(cache, cache->cc_nkeys, cur_skey); + return CatalogCacheComputeHashValue(cache, nkeys, v1, v2, v3, v4); +} + +/* + * CatalogCacheCompareTuple + * + * Compare a tuple to the passed arguments. + */ +static inline bool +CatalogCacheCompareTuple(const CatCache *cache, int nkeys, + const HeapTuple tuple, + const Datum *arguments) +{ + TupleDesc tupdesc = cache->cc_tupdesc; + const int *cc_key = cache->cc_key; + const CCFastEqualFN *cc_fastequal = cache->cc_fastequal; + int i; + + for (i = 0; i < nkeys; i++) + { + Datum atp; + bool isnull; + + /* + * XXX: might be worthwhile to only handle oid sysattr, to reduce + * overhead - it's the most common key. + */ + atp = heap_getattr(tuple, + cc_key[i], + tupdesc, + &isnull); + Assert(!isnull); + + if (!(cc_fastequal[i])(atp, arguments[i])) + { + return false; + } + } + return true; } @@ -878,7 +1006,8 @@ CatalogCacheInitializeCache(CatCache *cache) GetCCHashEqFuncs(keytype, &cache->cc_hashfunc[i], - &eqfunc); + &eqfunc, + &cache->cc_fastequal[i]); cache->cc_isname[i] = (keytype == NAMEOID); @@ -1020,7 +1149,7 @@ IndexScanOK(CatCache *cache, ScanKey cur_skey) } /* - * SearchCatCache + * SearchCatCacheInternal * * This call searches a system cache for a tuple, opening the relation * if necessary (on the first access to a particular cache). @@ -1042,15 +1171,64 @@ SearchCatCache(CatCache *cache, Datum v3, Datum v4) { - ScanKeyData cur_skey[CATCACHE_MAXKEYS]; + return SearchCatCacheInternal(cache, cache->cc_nkeys, v1, v2, v3, v4); +} + + +/* + * SearchCatCacheN() are SearchCatCache() versions for a specific number of + * arguments. The compiler can inline the body and unroll the loop, making + * them a bit faster than SearchCatCache(). + */ + +HeapTuple +SearchCatCache1(CatCache *cache, + Datum v1) +{ + return SearchCatCacheInternal(cache, 1, v1, 0, 0, 0); +} + + +HeapTuple +SearchCatCache2(CatCache *cache, + Datum v1, Datum v2) +{ + return SearchCatCacheInternal(cache, 2, v1, v2, 0, 0); +} + + +HeapTuple +SearchCatCache3(CatCache *cache, + Datum v1, Datum v2, Datum v3) +{ + return SearchCatCacheInternal(cache, 3, v1, v2, v3, 0); +} + + +HeapTuple +SearchCatCache4(CatCache *cache, + Datum v1, Datum v2, Datum v3, Datum v4) +{ + return SearchCatCacheInternal(cache, 4, v1, v2, v3, v4); +} + +/* + * Work-horse for SearchCatCache/SearchCatCacheN. + */ +static inline HeapTuple +SearchCatCacheInternal(CatCache *cache, + int nkeys, + Datum v1, + Datum v2, + Datum v3, + Datum v4) +{ + Datum arguments[CATCACHE_MAXKEYS]; uint32 hashValue; Index hashIndex; dlist_iter iter; dlist_head *bucket; CatCTup *ct; - Relation relation; - SysScanDesc scandesc; - HeapTuple ntp; /* Make sure we're in an xact, even if this ends up being a cache hit */ Assert(IsTransactionState()); @@ -1058,26 +1236,23 @@ SearchCatCache(CatCache *cache, /* * one-time startup overhead for each cache */ - if (cache->cc_tupdesc == NULL) + if (unlikely(cache->cc_tupdesc == NULL)) CatalogCacheInitializeCache(cache); #ifdef CATCACHE_STATS cache->cc_searches++; #endif - /* - * initialize the search key information - */ - memcpy(cur_skey, cache->cc_skey, sizeof(cur_skey)); - cur_skey[0].sk_argument = v1; - cur_skey[1].sk_argument = v2; - cur_skey[2].sk_argument = v3; - cur_skey[3].sk_argument = v4; + /* Initialize local parameter array */ + arguments[0] = v1; + arguments[1] = v2; + arguments[2] = v3; + arguments[3] = v4; /* * find the hash bucket in which to look for the tuple */ - hashValue = CatalogCacheComputeHashValue(cache, cache->cc_nkeys, cur_skey); + hashValue = CatalogCacheComputeHashValue(cache, nkeys, v1, v2, v3, v4); hashIndex = HASH_INDEX(hashValue, cache->cc_nbuckets); /* @@ -1089,8 +1264,6 @@ SearchCatCache(CatCache *cache, bucket = &cache->cc_bucket[hashIndex]; dlist_foreach(iter, bucket) { - bool res; - ct = dlist_container(CatCTup, cache_elem, iter.cur); if (ct->dead) @@ -1099,15 +1272,7 @@ SearchCatCache(CatCache *cache, if (ct->hash_value != hashValue) continue; /* quickly skip entry if wrong hash val */ - /* - * see if the cached tuple matches our key. - */ - HeapKeyTest(&ct->tuple, - cache->cc_tupdesc, - cache->cc_nkeys, - cur_skey, - res); - if (!res) + if (!CatalogCacheCompareTuple(cache, nkeys, &ct->tuple, arguments)) continue; /* @@ -1150,6 +1315,42 @@ SearchCatCache(CatCache *cache, } } + return SearchCatCacheMiss(cache, nkeys, hashValue, hashIndex, v1, v2, v3, v4); +} + +/* + * Search the actual catalogs, rather than the cache. + * + * This is kept separate from SearchCatCacheInternal() to keep the fast-path + * as small as possible. To avoid that effort being undone, try to explicitly + * forbid inlining. + */ +static pg_noinline HeapTuple +SearchCatCacheMiss(CatCache *cache, + int nkeys, + uint32 hashValue, + Index hashIndex, + Datum v1, + Datum v2, + Datum v3, + Datum v4) +{ + ScanKeyData cur_skey[CATCACHE_MAXKEYS]; + Relation relation; + SysScanDesc scandesc; + HeapTuple ntp; + CatCTup *ct; + + /* + * Ok, need to make a lookup in the relation, copy the scankey and fill out + * any per-call fields. + */ + memcpy(cur_skey, cache->cc_skey, sizeof(ScanKeyData) * nkeys); + cur_skey[0].sk_argument = v1; + cur_skey[1].sk_argument = v2; + cur_skey[2].sk_argument = v3; + cur_skey[3].sk_argument = v4; + /* * Tuple was not found in cache, so we have to try to retrieve it directly * from the relation. If found, we will add it to the cache; if not @@ -1171,7 +1372,7 @@ SearchCatCache(CatCache *cache, cache->cc_indexoid, IndexScanOK(cache, cur_skey), NULL, - cache->cc_nkeys, + nkeys, cur_skey); ct = NULL; @@ -1207,7 +1408,7 @@ SearchCatCache(CatCache *cache, if (IsBootstrapProcessingMode()) return NULL; - ntp = build_dummy_tuple(cache, cache->cc_nkeys, cur_skey); + ntp = build_dummy_tuple(cache, nkeys, cur_skey); ct = CatalogCacheCreateEntry(cache, ntp, hashValue, hashIndex, true); @@ -1288,27 +1489,16 @@ GetCatCacheHashValue(CatCache *cache, Datum v3, Datum v4) { - ScanKeyData cur_skey[CATCACHE_MAXKEYS]; - /* * one-time startup overhead for each cache */ if (cache->cc_tupdesc == NULL) CatalogCacheInitializeCache(cache); - /* - * initialize the search key information - */ - memcpy(cur_skey, cache->cc_skey, sizeof(cur_skey)); - cur_skey[0].sk_argument = v1; - cur_skey[1].sk_argument = v2; - cur_skey[2].sk_argument = v3; - cur_skey[3].sk_argument = v4; - /* * calculate the hash value */ - return CatalogCacheComputeHashValue(cache, cache->cc_nkeys, cur_skey); + return CatalogCacheComputeHashValue(cache, cache->cc_nkeys, v1, v2, v3, v4); } @@ -1329,7 +1519,7 @@ SearchCatCacheList(CatCache *cache, Datum v3, Datum v4) { - ScanKeyData cur_skey[CATCACHE_MAXKEYS]; + Datum arguments[CATCACHE_MAXKEYS]; uint32 lHashValue; dlist_iter iter; CatCList *cl; @@ -1354,21 +1544,18 @@ SearchCatCacheList(CatCache *cache, cache->cc_lsearches++; #endif - /* - * initialize the search key information - */ - memcpy(cur_skey, cache->cc_skey, sizeof(cur_skey)); - cur_skey[0].sk_argument = v1; - cur_skey[1].sk_argument = v2; - cur_skey[2].sk_argument = v3; - cur_skey[3].sk_argument = v4; + /* Initialize local parameter array */ + arguments[0] = v1; + arguments[1] = v2; + arguments[2] = v3; + arguments[3] = v4; /* * compute a hash value of the given keys for faster search. We don't * presently divide the CatCList items into buckets, but this still lets * us skip non-matching items quickly most of the time. */ - lHashValue = CatalogCacheComputeHashValue(cache, nkeys, cur_skey); + lHashValue = CatalogCacheComputeHashValue(cache, nkeys, v1, v2, v3, v4); /* * scan the items until we find a match or exhaust our list @@ -1378,8 +1565,6 @@ SearchCatCacheList(CatCache *cache, */ dlist_foreach(iter, &cache->cc_lists) { - bool res; - cl = dlist_container(CatCList, cache_elem, iter.cur); if (cl->dead) @@ -1393,12 +1578,8 @@ SearchCatCacheList(CatCache *cache, */ if (cl->nkeys != nkeys) continue; - HeapKeyTest(&cl->tuple, - cache->cc_tupdesc, - nkeys, - cur_skey, - res); - if (!res) + + if (!CatalogCacheCompareTuple(cache, nkeys, &cl->tuple, arguments)) continue; /* @@ -1441,9 +1622,20 @@ SearchCatCacheList(CatCache *cache, PG_TRY(); { + ScanKeyData cur_skey[CATCACHE_MAXKEYS]; Relation relation; SysScanDesc scandesc; + /* + * Ok, need to make a lookup in the relation, copy the scankey and fill out + * any per-call fields. + */ + memcpy(cur_skey, cache->cc_skey, sizeof(ScanKeyData) * cache->cc_nkeys); + cur_skey[0].sk_argument = v1; + cur_skey[1].sk_argument = v2; + cur_skey[2].sk_argument = v3; + cur_skey[3].sk_argument = v4; + relation = heap_open(cache->cc_reloid, AccessShareLock); scandesc = systable_beginscan(relation, @@ -1467,7 +1659,7 @@ SearchCatCacheList(CatCache *cache, * See if there's an entry for this tuple already. */ ct = NULL; - hashValue = CatalogCacheComputeTupleHashValue(cache, ntp); + hashValue = CatalogCacheComputeTupleHashValue(cache, cache->cc_nkeys, ntp); hashIndex = HASH_INDEX(hashValue, cache->cc_nbuckets); bucket = &cache->cc_bucket[hashIndex]; @@ -1820,7 +2012,7 @@ PrepareToInvalidateCacheTuple(Relation relation, if (ccp->cc_tupdesc == NULL) CatalogCacheInitializeCache(ccp); - hashvalue = CatalogCacheComputeTupleHashValue(ccp, tuple); + hashvalue = CatalogCacheComputeTupleHashValue(ccp, ccp->cc_nkeys, tuple); dbid = ccp->cc_relisshared ? (Oid) 0 : MyDatabaseId; (*function) (ccp->id, hashvalue, dbid); @@ -1829,7 +2021,7 @@ PrepareToInvalidateCacheTuple(Relation relation, { uint32 newhashvalue; - newhashvalue = CatalogCacheComputeTupleHashValue(ccp, newtuple); + newhashvalue = CatalogCacheComputeTupleHashValue(ccp, ccp->cc_nkeys, newtuple); if (newhashvalue != hashvalue) (*function) (ccp->id, newhashvalue, dbid); diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index fcbb683a99..888edbb325 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -1102,13 +1102,56 @@ SearchSysCache(int cacheId, Datum key3, Datum key4) { - if (cacheId < 0 || cacheId >= SysCacheSize || - !PointerIsValid(SysCache[cacheId])) - elog(ERROR, "invalid cache ID: %d", cacheId); + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); return SearchCatCache(SysCache[cacheId], key1, key2, key3, key4); } +HeapTuple +SearchSysCache1(int cacheId, + Datum key1) +{ + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); + Assert(SysCache[cacheId]->cc_nkeys == 1); + + return SearchCatCache1(SysCache[cacheId], key1); +} + +HeapTuple +SearchSysCache2(int cacheId, + Datum key1, Datum key2) +{ + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); + Assert(SysCache[cacheId]->cc_nkeys == 2); + + return SearchCatCache2(SysCache[cacheId], key1, key2); +} + +HeapTuple +SearchSysCache3(int cacheId, + Datum key1, Datum key2, Datum key3) +{ + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); + Assert(SysCache[cacheId]->cc_nkeys == 3); + + return SearchCatCache3(SysCache[cacheId], key1, key2, key3); +} + +HeapTuple +SearchSysCache4(int cacheId, + Datum key1, Datum key2, Datum key3, Datum key4) +{ + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); + Assert(SysCache[cacheId]->cc_nkeys == 4); + + return SearchCatCache4(SysCache[cacheId], key1, key2, key3, key4); +} + /* * ReleaseSysCache * Release previously grabbed reference count on a tuple diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index 200a3022e7..360f0c5dd5 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -34,6 +34,10 @@ #define CATCACHE_MAXKEYS 4 + +typedef uint32 (*CCHashFN) (Datum datum); +typedef bool (*CCFastEqualFN) (Datum a, Datum b); + typedef struct catcache { int id; /* cache identifier --- see syscache.h */ @@ -47,7 +51,8 @@ typedef struct catcache int cc_nbuckets; /* # of hash buckets in this cache */ int cc_nkeys; /* # of keys (1..CATCACHE_MAXKEYS) */ int cc_key[CATCACHE_MAXKEYS]; /* AttrNumber of each key */ - PGFunction cc_hashfunc[CATCACHE_MAXKEYS]; /* hash function for each key */ + CCHashFN cc_hashfunc[CATCACHE_MAXKEYS]; /* hash function for each key */ + CCFastEqualFN cc_fastequal[CATCACHE_MAXKEYS]; /* fast equal function for each key */ ScanKeyData cc_skey[CATCACHE_MAXKEYS]; /* precomputed key info for heap * scans */ bool cc_isname[CATCACHE_MAXKEYS]; /* flag "name" key columns */ @@ -174,8 +179,15 @@ extern CatCache *InitCatCache(int id, Oid reloid, Oid indexoid, extern void InitCatCachePhase2(CatCache *cache, bool touch_index); extern HeapTuple SearchCatCache(CatCache *cache, - Datum v1, Datum v2, - Datum v3, Datum v4); + Datum v1, Datum v2, Datum v3, Datum v4); +extern HeapTuple SearchCatCache1(CatCache *cache, + Datum v1); +extern HeapTuple SearchCatCache2(CatCache *cache, + Datum v1, Datum v2); +extern HeapTuple SearchCatCache3(CatCache *cache, + Datum v1, Datum v2, Datum v3); +extern HeapTuple SearchCatCache4(CatCache *cache, + Datum v1, Datum v2, Datum v3, Datum v4); extern void ReleaseCatCache(HeapTuple tuple); extern uint32 GetCatCacheHashValue(CatCache *cache, diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h index 8a92ea27ac..12bda02cd7 100644 --- a/src/include/utils/syscache.h +++ b/src/include/utils/syscache.h @@ -117,6 +117,20 @@ extern void InitCatalogCachePhase2(void); extern HeapTuple SearchSysCache(int cacheId, Datum key1, Datum key2, Datum key3, Datum key4); + +/* + * The use of argument specific numbers is encouraged, they're faster, and + * insulates the caller from changes in the maximum number of keys. + */ +extern HeapTuple SearchSysCache1(int cacheId, + Datum key1); +extern HeapTuple SearchSysCache2(int cacheId, + Datum key1, Datum key2); +extern HeapTuple SearchSysCache3(int cacheId, + Datum key1, Datum key2, Datum key3); +extern HeapTuple SearchSysCache4(int cacheId, + Datum key1, Datum key2, Datum key3, Datum key4); + extern void ReleaseSysCache(HeapTuple tuple); /* convenience routines */ @@ -156,15 +170,6 @@ extern bool RelationSupportsSysCache(Oid relid); * functions is encouraged, as it insulates the caller from changes in the * maximum number of keys. */ -#define SearchSysCache1(cacheId, key1) \ - SearchSysCache(cacheId, key1, 0, 0, 0) -#define SearchSysCache2(cacheId, key1, key2) \ - SearchSysCache(cacheId, key1, key2, 0, 0) -#define SearchSysCache3(cacheId, key1, key2, key3) \ - SearchSysCache(cacheId, key1, key2, key3, 0) -#define SearchSysCache4(cacheId, key1, key2, key3, key4) \ - SearchSysCache(cacheId, key1, key2, key3, key4) - #define SearchSysCacheCopy1(cacheId, key1) \ SearchSysCacheCopy(cacheId, key1, 0, 0, 0) #define SearchSysCacheCopy2(cacheId, key1, key2) \ -- 2.14.1.536.g6867272d5b.dirty --o6fgx62zvq657xjz Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0008-WIP-Improve-getBaseTypeAndTypemod-performance-for-v1.patch" ^ permalink raw reply [nested|flat] 6+ messages in thread
* [PATCH 3/3] Improve sys/catcache performance. @ 2017-09-12 01:25 Andres Freund <[email protected]> 0 siblings, 0 replies; 6+ messages in thread From: Andres Freund @ 2017-09-12 01:25 UTC (permalink / raw) The following are the individual improvements: 1) Avoidance of FunctionCallInfo based function calls, replaced by more efficient functions with a native C argument interface. 2) Don't extract columns from a cache entry's tuple whenever matching entries - instead store them as a Datum array. This also allows to get rid of having to build dummy tuples for negative & list entries, and of a hack for dealing with cstring vs. text weirdness. 3) Reorder members of catcache.h struct, so imortant entries are more likely to be on one cacheline. 4) Allowing the compiler to specialize critical SearchCatCache for a specific number of attributes allows to unroll loops and avoid other nkeys dependant initialization. 5) Only initializing the ScanKey when necessary, i.e. catcache misses, greatly reduces cache unnecessary cpu cache misses. 6) Split of the cache-miss case from the hash lookup, reducing stack allocations etc in the common case. 7) CatCTup and their corresponding heaptuple are allocated in one piece. This results in making cache lookups themselves roughly three times as fast - full-system benchmarks obviously improve less than that. I've also evaluated further techniques: - replace open coded hash with simplehash - the list walk right now shows up in profiles. Unfortunately it's not easy to do so safely as an entry's memory location can change at various times, which doesn't work well with the refcounting and cache invalidation. - Cacheline-aligning CatCTup entries - helps some with performance, but the win isn't big and the code for it is ugly, because the tuples have to be freed as well. - add more proper functions, rather than macros for SearchSysCacheCopyN etc., but right now they don't show up in profiles. The reason the macro wrapper for syscache.c/h have to be changed, rather than just catcache, is that doing otherwise would require exposing the SysCache array to the outside. That might be a good idea anyway, but it's for another day. Author: Andres Freund Discussion: https://postgr.es/m/[email protected] --- src/backend/utils/cache/catcache.c | 678 +++++++++++++++++++++++++------------ src/backend/utils/cache/syscache.c | 49 ++- src/include/utils/catcache.h | 122 ++++--- src/include/utils/syscache.h | 23 +- 4 files changed, 592 insertions(+), 280 deletions(-) diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c index e092801025..4babcbfcaf 100644 --- a/src/backend/utils/cache/catcache.c +++ b/src/backend/utils/cache/catcache.c @@ -30,7 +30,9 @@ #endif #include "storage/lmgr.h" #include "utils/builtins.h" +#include "utils/datum.h" #include "utils/fmgroids.h" +#include "utils/hashutils.h" #include "utils/inval.h" #include "utils/memutils.h" #include "utils/rel.h" @@ -72,11 +74,25 @@ /* Cache management header --- pointer is NULL until created */ static CatCacheHeader *CacheHdr = NULL; +static inline HeapTuple SearchCatCacheInternal(CatCache *cache, + int nkeys, + Datum v1, Datum v2, + Datum v3, Datum v4); + +static pg_noinline HeapTuple SearchCatCacheMiss(CatCache *cache, + int nkeys, + uint32 hashValue, + Index hashIndex, + Datum v1, Datum v2, + Datum v3, Datum v4); static uint32 CatalogCacheComputeHashValue(CatCache *cache, int nkeys, - ScanKey cur_skey); -static uint32 CatalogCacheComputeTupleHashValue(CatCache *cache, + Datum v1, Datum v2, Datum v3, Datum v4); +static uint32 CatalogCacheComputeTupleHashValue(CatCache *cache, int nkeys, HeapTuple tuple); +static inline bool CatalogCacheCompareTuple(const CatCache *cache, int nkeys, + const Datum *cachekeys, + const Datum *searchkeys); #ifdef CATCACHE_STATS static void CatCachePrintStats(int code, Datum arg); @@ -85,9 +101,14 @@ static void CatCacheRemoveCTup(CatCache *cache, CatCTup *ct); static void CatCacheRemoveCList(CatCache *cache, CatCList *cl); static void CatalogCacheInitializeCache(CatCache *cache); static CatCTup *CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp, + Datum *arguments, uint32 hashValue, Index hashIndex, bool negative); -static HeapTuple build_dummy_tuple(CatCache *cache, int nkeys, ScanKey skeys); + +static void CatCacheFreeKeys(TupleDesc tupdesc, int nkeys, int *attnos, + Datum *keys); +static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, + Datum *srckeys, Datum *dstkeys); /* @@ -95,45 +116,127 @@ static HeapTuple build_dummy_tuple(CatCache *cache, int nkeys, ScanKey skeys); */ /* - * Look up the hash and equality functions for system types that are used - * as cache key fields. - * - * XXX this should be replaced by catalog lookups, - * but that seems to pose considerable risk of circularity... + * Hash and equality functions for system types that are used as cache key + * fields. To compute hashes, and to check for hash collisions, use functions + * hardcoded for that purpose. This is sufficiently performance critical that + * the overhead of SQL style function calls is noticeable. */ + +static bool +chareqfast(Datum a, Datum b) +{ + return DatumGetChar(a) == DatumGetChar(b); +} + +static uint32 +charhashfast(Datum datum) +{ + return murmurhash32((int32) DatumGetChar(datum)); +} + +static bool +nameeqfast(Datum a, Datum b) +{ + char *ca = NameStr(*DatumGetName(a)); + char *cb = NameStr(*DatumGetName(b)); + + return strncmp(ca, cb, NAMEDATALEN) == 0; +} + +static uint32 +namehashfast(Datum datum) +{ + char *key = NameStr(*DatumGetName(datum)); + + return hash_any((unsigned char *) key, strlen(key)); +} + +static bool +int2eqfast(Datum a, Datum b) +{ + return DatumGetInt16(a) == DatumGetInt16(b); +} + +static uint32 +int2hashfast(Datum datum) +{ + return murmurhash32((int32) DatumGetInt16(datum)); +} + +static bool +int4eqfast(Datum a, Datum b) +{ + return DatumGetInt32(a) == DatumGetInt32(b); +} + +static uint32 +int4hashfast(Datum datum) +{ + return murmurhash32((int32) DatumGetInt32(datum)); +} + +static bool +texteqfast(Datum a, Datum b) +{ + /* not as performance critical & "complicated" */ + return DatumGetBool(DirectFunctionCall2(texteq, a, b)); +} + +static uint32 +texthashfast(Datum datum) +{ + /* not as performance critical & "complicated" */ + return DatumGetInt32(DirectFunctionCall1(hashtext, datum)); +} + +static bool +oidvectoreqfast(Datum a, Datum b) +{ + /* not as performance critical & "complicated" */ + return DatumGetBool(DirectFunctionCall2(oidvectoreq, a, b)); +} + +static uint32 +oidvectorhashfast(Datum datum) +{ + /* not as performance critical & "complicated" */ + return DatumGetInt32(DirectFunctionCall1(hashoidvector, datum)); +} + +/* Lookup support functions for a type. */ static void -GetCCHashEqFuncs(Oid keytype, PGFunction *hashfunc, RegProcedure *eqfunc) +GetCCHashEqFuncs(Oid keytype, CCHashFN *hashfunc, RegProcedure *eqfunc, CCFastEqualFN *fasteqfunc) { switch (keytype) { case BOOLOID: - *hashfunc = hashchar; - + *hashfunc = charhashfast; + *fasteqfunc = chareqfast; *eqfunc = F_BOOLEQ; break; case CHAROID: - *hashfunc = hashchar; - + *hashfunc = charhashfast; + *fasteqfunc = chareqfast; *eqfunc = F_CHAREQ; break; case NAMEOID: - *hashfunc = hashname; - + *hashfunc = namehashfast; + *fasteqfunc = nameeqfast; *eqfunc = F_NAMEEQ; break; case INT2OID: - *hashfunc = hashint2; - + *hashfunc = int2hashfast; + *fasteqfunc = int2eqfast; *eqfunc = F_INT2EQ; break; case INT4OID: - *hashfunc = hashint4; - + *hashfunc = int4hashfast; + *fasteqfunc = int4eqfast; *eqfunc = F_INT4EQ; break; case TEXTOID: - *hashfunc = hashtext; - + *hashfunc = texthashfast; + *fasteqfunc = texteqfast; *eqfunc = F_TEXTEQ; break; case OIDOID: @@ -147,13 +250,13 @@ GetCCHashEqFuncs(Oid keytype, PGFunction *hashfunc, RegProcedure *eqfunc) case REGDICTIONARYOID: case REGROLEOID: case REGNAMESPACEOID: - *hashfunc = hashoid; - + *hashfunc = int4hashfast; + *fasteqfunc = int4eqfast; *eqfunc = F_OIDEQ; break; case OIDVECTOROID: - *hashfunc = hashoidvector; - + *hashfunc = oidvectorhashfast; + *fasteqfunc = oidvectoreqfast; *eqfunc = F_OIDVECTOREQ; break; default: @@ -171,10 +274,12 @@ GetCCHashEqFuncs(Oid keytype, PGFunction *hashfunc, RegProcedure *eqfunc) * Compute the hash value associated with a given set of lookup keys */ static uint32 -CatalogCacheComputeHashValue(CatCache *cache, int nkeys, ScanKey cur_skey) +CatalogCacheComputeHashValue(CatCache *cache, int nkeys, + Datum v1, Datum v2, Datum v3, Datum v4) { uint32 hashValue = 0; uint32 oneHash; + CCHashFN *cc_hashfunc = cache->cc_hashfunc; CACHE4_elog(DEBUG2, "CatalogCacheComputeHashValue %s %d %p", cache->cc_relname, @@ -184,30 +289,26 @@ CatalogCacheComputeHashValue(CatCache *cache, int nkeys, ScanKey cur_skey) switch (nkeys) { case 4: - oneHash = - DatumGetUInt32(DirectFunctionCall1(cache->cc_hashfunc[3], - cur_skey[3].sk_argument)); + oneHash = (cc_hashfunc[3])(v4); + hashValue ^= oneHash << 24; hashValue ^= oneHash >> 8; /* FALLTHROUGH */ case 3: - oneHash = - DatumGetUInt32(DirectFunctionCall1(cache->cc_hashfunc[2], - cur_skey[2].sk_argument)); + oneHash = (cc_hashfunc[2])(v3); + hashValue ^= oneHash << 16; hashValue ^= oneHash >> 16; /* FALLTHROUGH */ case 2: - oneHash = - DatumGetUInt32(DirectFunctionCall1(cache->cc_hashfunc[1], - cur_skey[1].sk_argument)); + oneHash = (cc_hashfunc[1])(v2); + hashValue ^= oneHash << 8; hashValue ^= oneHash >> 24; /* FALLTHROUGH */ case 1: - oneHash = - DatumGetUInt32(DirectFunctionCall1(cache->cc_hashfunc[0], - cur_skey[0].sk_argument)); + oneHash = (cc_hashfunc[0])(v1); + hashValue ^= oneHash; break; default: @@ -224,63 +325,81 @@ CatalogCacheComputeHashValue(CatCache *cache, int nkeys, ScanKey cur_skey) * Compute the hash value associated with a given tuple to be cached */ static uint32 -CatalogCacheComputeTupleHashValue(CatCache *cache, HeapTuple tuple) +CatalogCacheComputeTupleHashValue(CatCache *cache, int nkeys, HeapTuple tuple) { - ScanKeyData cur_skey[CATCACHE_MAXKEYS]; + Datum v1 = 0, v2 = 0, v3 = 0, v4 = 0; bool isNull = false; - - /* Copy pre-initialized overhead data for scankey */ - memcpy(cur_skey, cache->cc_skey, sizeof(cur_skey)); + int *cc_keyno = cache->cc_keyno; + TupleDesc cc_tupdesc = cache->cc_tupdesc; /* Now extract key fields from tuple, insert into scankey */ - switch (cache->cc_nkeys) + switch (nkeys) { case 4: - cur_skey[3].sk_argument = - (cache->cc_key[3] == ObjectIdAttributeNumber) + v4 = (cc_keyno[3] == ObjectIdAttributeNumber) ? ObjectIdGetDatum(HeapTupleGetOid(tuple)) : fastgetattr(tuple, - cache->cc_key[3], - cache->cc_tupdesc, + cc_keyno[3], + cc_tupdesc, &isNull); Assert(!isNull); /* FALLTHROUGH */ case 3: - cur_skey[2].sk_argument = - (cache->cc_key[2] == ObjectIdAttributeNumber) + v3 = (cc_keyno[2] == ObjectIdAttributeNumber) ? ObjectIdGetDatum(HeapTupleGetOid(tuple)) : fastgetattr(tuple, - cache->cc_key[2], - cache->cc_tupdesc, + cc_keyno[2], + cc_tupdesc, &isNull); Assert(!isNull); /* FALLTHROUGH */ case 2: - cur_skey[1].sk_argument = - (cache->cc_key[1] == ObjectIdAttributeNumber) + v2 = (cc_keyno[1] == ObjectIdAttributeNumber) ? ObjectIdGetDatum(HeapTupleGetOid(tuple)) : fastgetattr(tuple, - cache->cc_key[1], - cache->cc_tupdesc, + cc_keyno[1], + cc_tupdesc, &isNull); Assert(!isNull); /* FALLTHROUGH */ case 1: - cur_skey[0].sk_argument = - (cache->cc_key[0] == ObjectIdAttributeNumber) + v1 = (cc_keyno[0] == ObjectIdAttributeNumber) ? ObjectIdGetDatum(HeapTupleGetOid(tuple)) : fastgetattr(tuple, - cache->cc_key[0], - cache->cc_tupdesc, + cc_keyno[0], + cc_tupdesc, &isNull); Assert(!isNull); break; default: - elog(FATAL, "wrong number of hash keys: %d", cache->cc_nkeys); + elog(FATAL, "wrong number of hash keys: %d", nkeys); break; } - return CatalogCacheComputeHashValue(cache, cache->cc_nkeys, cur_skey); + return CatalogCacheComputeHashValue(cache, nkeys, v1, v2, v3, v4); +} + +/* + * CatalogCacheCompareTuple + * + * Compare a tuple to the passed arguments. + */ +static inline bool +CatalogCacheCompareTuple(const CatCache *cache, int nkeys, + const Datum *cachekeys, + const Datum *searchkeys) +{ + const CCFastEqualFN *cc_fastequal = cache->cc_fastequal; + int i; + + for (i = 0; i < nkeys; i++) + { + if (!(cc_fastequal[i])(cachekeys[i], searchkeys[i])) + { + return false; + } + } + return true; } @@ -371,9 +490,14 @@ CatCacheRemoveCTup(CatCache *cache, CatCTup *ct) /* delink from linked list */ dlist_delete(&ct->cache_elem); - /* free associated tuple data */ - if (ct->tuple.t_data != NULL) - pfree(ct->tuple.t_data); + /* + * Free keys when we're dealing with a negative entry, normal entries just + * point into tuple, allocated together with the CatCTup. + */ + if (ct->negative) + CatCacheFreeKeys(cache->cc_tupdesc, cache->cc_nkeys, + cache->cc_keyno, ct->keys); + pfree(ct); --cache->cc_ntup; @@ -414,9 +538,10 @@ CatCacheRemoveCList(CatCache *cache, CatCList *cl) /* delink from linked list */ dlist_delete(&cl->cache_elem); - /* free associated tuple data */ - if (cl->tuple.t_data != NULL) - pfree(cl->tuple.t_data); + /* free associated column data */ + CatCacheFreeKeys(cache->cc_tupdesc, cl->nkeys, + cache->cc_keyno, cl->keys); + pfree(cl); } @@ -660,6 +785,7 @@ InitCatCache(int id, { CatCache *cp; MemoryContext oldcxt; + size_t sz; int i; /* @@ -699,11 +825,12 @@ InitCatCache(int id, } /* - * allocate a new cache structure + * Allocate a new cache structure, aligning to a cacheline boundary * * Note: we rely on zeroing to initialize all the dlist headers correctly */ - cp = (CatCache *) palloc0(sizeof(CatCache)); + sz = sizeof(CatCache) + PG_CACHE_LINE_SIZE; + cp = (CatCache *) CACHELINEALIGN(palloc0(sz)); cp->cc_bucket = palloc0(nbuckets * sizeof(dlist_head)); /* @@ -721,7 +848,7 @@ InitCatCache(int id, cp->cc_nbuckets = nbuckets; cp->cc_nkeys = nkeys; for (i = 0; i < nkeys; ++i) - cp->cc_key[i] = key[i]; + cp->cc_keyno[i] = key[i]; /* * new cache is initialized as far as we can go for now. print some @@ -794,13 +921,13 @@ RehashCatCache(CatCache *cp) #define CatalogCacheInitializeCache_DEBUG2 \ do { \ - if (cache->cc_key[i] > 0) { \ + if (cache->cc_keyno[i] > 0) { \ elog(DEBUG2, "CatalogCacheInitializeCache: load %d/%d w/%d, %u", \ - i+1, cache->cc_nkeys, cache->cc_key[i], \ - TupleDescAttr(tupdesc, cache->cc_key[i] - 1)->atttypid); \ + i+1, cache->cc_nkeys, cache->cc_keyno[i], \ + TupleDescAttr(tupdesc, cache->cc_keyno[i] - 1)->atttypid); \ } else { \ elog(DEBUG2, "CatalogCacheInitializeCache: load %d/%d w/%d", \ - i+1, cache->cc_nkeys, cache->cc_key[i]); \ + i+1, cache->cc_nkeys, cache->cc_keyno[i]); \ } \ } while(0) #else @@ -860,10 +987,10 @@ CatalogCacheInitializeCache(CatCache *cache) CatalogCacheInitializeCache_DEBUG2; - if (cache->cc_key[i] > 0) + if (cache->cc_keyno[i] > 0) { Form_pg_attribute attr = TupleDescAttr(tupdesc, - cache->cc_key[i] - 1); + cache->cc_keyno[i] - 1); keytype = attr->atttypid; /* cache key columns should always be NOT NULL */ @@ -871,16 +998,15 @@ CatalogCacheInitializeCache(CatCache *cache) } else { - if (cache->cc_key[i] != ObjectIdAttributeNumber) + if (cache->cc_keyno[i] != ObjectIdAttributeNumber) elog(FATAL, "only sys attr supported in caches is OID"); keytype = OIDOID; } GetCCHashEqFuncs(keytype, &cache->cc_hashfunc[i], - &eqfunc); - - cache->cc_isname[i] = (keytype == NAMEOID); + &eqfunc, + &cache->cc_fastequal[i]); /* * Do equality-function lookup (we assume this won't need a catalog @@ -891,7 +1017,7 @@ CatalogCacheInitializeCache(CatCache *cache) CacheMemoryContext); /* Initialize sk_attno suitably for HeapKeyTest() and heap scans */ - cache->cc_skey[i].sk_attno = cache->cc_key[i]; + cache->cc_skey[i].sk_attno = cache->cc_keyno[i]; /* Fill in sk_strategy as well --- always standard equality */ cache->cc_skey[i].sk_strategy = BTEqualStrategyNumber; @@ -1020,7 +1146,7 @@ IndexScanOK(CatCache *cache, ScanKey cur_skey) } /* - * SearchCatCache + * SearchCatCacheInternal * * This call searches a system cache for a tuple, opening the relation * if necessary (on the first access to a particular cache). @@ -1042,42 +1168,90 @@ SearchCatCache(CatCache *cache, Datum v3, Datum v4) { - ScanKeyData cur_skey[CATCACHE_MAXKEYS]; + return SearchCatCacheInternal(cache, cache->cc_nkeys, v1, v2, v3, v4); +} + + +/* + * SearchCatCacheN() are SearchCatCache() versions for a specific number of + * arguments. The compiler can inline the body and unroll the loop, making + * them a bit faster than SearchCatCache(). + */ + +HeapTuple +SearchCatCache1(CatCache *cache, + Datum v1) +{ + return SearchCatCacheInternal(cache, 1, v1, 0, 0, 0); +} + + +HeapTuple +SearchCatCache2(CatCache *cache, + Datum v1, Datum v2) +{ + return SearchCatCacheInternal(cache, 2, v1, v2, 0, 0); +} + + +HeapTuple +SearchCatCache3(CatCache *cache, + Datum v1, Datum v2, Datum v3) +{ + return SearchCatCacheInternal(cache, 3, v1, v2, v3, 0); +} + + +HeapTuple +SearchCatCache4(CatCache *cache, + Datum v1, Datum v2, Datum v3, Datum v4) +{ + return SearchCatCacheInternal(cache, 4, v1, v2, v3, v4); +} + +/* + * Work-horse for SearchCatCache/SearchCatCacheN. + */ +static inline HeapTuple +SearchCatCacheInternal(CatCache *cache, + int nkeys, + Datum v1, + Datum v2, + Datum v3, + Datum v4) +{ + Datum arguments[CATCACHE_MAXKEYS]; uint32 hashValue; Index hashIndex; dlist_iter iter; dlist_head *bucket; CatCTup *ct; - Relation relation; - SysScanDesc scandesc; - HeapTuple ntp; /* Make sure we're in an xact, even if this ends up being a cache hit */ Assert(IsTransactionState()); + Assert(cache->cc_nkeys == nkeys); + /* * one-time startup overhead for each cache */ - if (cache->cc_tupdesc == NULL) + if (unlikely(cache->cc_tupdesc == NULL)) CatalogCacheInitializeCache(cache); #ifdef CATCACHE_STATS cache->cc_searches++; #endif - /* - * initialize the search key information - */ - memcpy(cur_skey, cache->cc_skey, sizeof(cur_skey)); - cur_skey[0].sk_argument = v1; - cur_skey[1].sk_argument = v2; - cur_skey[2].sk_argument = v3; - cur_skey[3].sk_argument = v4; + /* Initialize local parameter array */ + arguments[0] = v1; + arguments[1] = v2; + arguments[2] = v3; + arguments[3] = v4; /* * find the hash bucket in which to look for the tuple */ - hashValue = CatalogCacheComputeHashValue(cache, cache->cc_nkeys, cur_skey); + hashValue = CatalogCacheComputeHashValue(cache, nkeys, v1, v2, v3, v4); hashIndex = HASH_INDEX(hashValue, cache->cc_nbuckets); /* @@ -1089,8 +1263,6 @@ SearchCatCache(CatCache *cache, bucket = &cache->cc_bucket[hashIndex]; dlist_foreach(iter, bucket) { - bool res; - ct = dlist_container(CatCTup, cache_elem, iter.cur); if (ct->dead) @@ -1099,15 +1271,7 @@ SearchCatCache(CatCache *cache, if (ct->hash_value != hashValue) continue; /* quickly skip entry if wrong hash val */ - /* - * see if the cached tuple matches our key. - */ - HeapKeyTest(&ct->tuple, - cache->cc_tupdesc, - cache->cc_nkeys, - cur_skey, - res); - if (!res) + if (!CatalogCacheCompareTuple(cache, nkeys, ct->keys, arguments)) continue; /* @@ -1150,6 +1314,49 @@ SearchCatCache(CatCache *cache, } } + return SearchCatCacheMiss(cache, nkeys, hashValue, hashIndex, v1, v2, v3, v4); +} + +/* + * Search the actual catalogs, rather than the cache. + * + * This is kept separate from SearchCatCacheInternal() to keep the fast-path + * as small as possible. To avoid that effort being undone by a helpful + * compiler, try to explicitly forbid inlining. + */ +static pg_noinline HeapTuple +SearchCatCacheMiss(CatCache *cache, + int nkeys, + uint32 hashValue, + Index hashIndex, + Datum v1, + Datum v2, + Datum v3, + Datum v4) +{ + ScanKeyData cur_skey[CATCACHE_MAXKEYS]; + Relation relation; + SysScanDesc scandesc; + HeapTuple ntp; + CatCTup *ct; + Datum arguments[CATCACHE_MAXKEYS]; + + /* Initialize local parameter array */ + arguments[0] = v1; + arguments[1] = v2; + arguments[2] = v3; + arguments[3] = v4; + + /* + * Ok, need to make a lookup in the relation, copy the scankey and fill out + * any per-call fields. + */ + memcpy(cur_skey, cache->cc_skey, sizeof(ScanKeyData) * nkeys); + cur_skey[0].sk_argument = v1; + cur_skey[1].sk_argument = v2; + cur_skey[2].sk_argument = v3; + cur_skey[3].sk_argument = v4; + /* * Tuple was not found in cache, so we have to try to retrieve it directly * from the relation. If found, we will add it to the cache; if not @@ -1171,14 +1378,14 @@ SearchCatCache(CatCache *cache, cache->cc_indexoid, IndexScanOK(cache, cur_skey), NULL, - cache->cc_nkeys, + nkeys, cur_skey); ct = NULL; while (HeapTupleIsValid(ntp = systable_getnext(scandesc))) { - ct = CatalogCacheCreateEntry(cache, ntp, + ct = CatalogCacheCreateEntry(cache, ntp, arguments, hashValue, hashIndex, false); /* immediately set the refcount to 1 */ @@ -1207,11 +1414,9 @@ SearchCatCache(CatCache *cache, if (IsBootstrapProcessingMode()) return NULL; - ntp = build_dummy_tuple(cache, cache->cc_nkeys, cur_skey); - ct = CatalogCacheCreateEntry(cache, ntp, + ct = CatalogCacheCreateEntry(cache, NULL, arguments, hashValue, hashIndex, true); - heap_freetuple(ntp); CACHE4_elog(DEBUG2, "SearchCatCache(%s): Contains %d/%d tuples", cache->cc_relname, cache->cc_ntup, CacheHdr->ch_ntup); @@ -1288,27 +1493,16 @@ GetCatCacheHashValue(CatCache *cache, Datum v3, Datum v4) { - ScanKeyData cur_skey[CATCACHE_MAXKEYS]; - /* * one-time startup overhead for each cache */ if (cache->cc_tupdesc == NULL) CatalogCacheInitializeCache(cache); - /* - * initialize the search key information - */ - memcpy(cur_skey, cache->cc_skey, sizeof(cur_skey)); - cur_skey[0].sk_argument = v1; - cur_skey[1].sk_argument = v2; - cur_skey[2].sk_argument = v3; - cur_skey[3].sk_argument = v4; - /* * calculate the hash value */ - return CatalogCacheComputeHashValue(cache, cache->cc_nkeys, cur_skey); + return CatalogCacheComputeHashValue(cache, cache->cc_nkeys, v1, v2, v3, v4); } @@ -1329,7 +1523,7 @@ SearchCatCacheList(CatCache *cache, Datum v3, Datum v4) { - ScanKeyData cur_skey[CATCACHE_MAXKEYS]; + Datum arguments[CATCACHE_MAXKEYS]; uint32 lHashValue; dlist_iter iter; CatCList *cl; @@ -1354,21 +1548,18 @@ SearchCatCacheList(CatCache *cache, cache->cc_lsearches++; #endif - /* - * initialize the search key information - */ - memcpy(cur_skey, cache->cc_skey, sizeof(cur_skey)); - cur_skey[0].sk_argument = v1; - cur_skey[1].sk_argument = v2; - cur_skey[2].sk_argument = v3; - cur_skey[3].sk_argument = v4; + /* Initialize local parameter array */ + arguments[0] = v1; + arguments[1] = v2; + arguments[2] = v3; + arguments[3] = v4; /* * compute a hash value of the given keys for faster search. We don't * presently divide the CatCList items into buckets, but this still lets * us skip non-matching items quickly most of the time. */ - lHashValue = CatalogCacheComputeHashValue(cache, nkeys, cur_skey); + lHashValue = CatalogCacheComputeHashValue(cache, nkeys, v1, v2, v3, v4); /* * scan the items until we find a match or exhaust our list @@ -1378,8 +1569,6 @@ SearchCatCacheList(CatCache *cache, */ dlist_foreach(iter, &cache->cc_lists) { - bool res; - cl = dlist_container(CatCList, cache_elem, iter.cur); if (cl->dead) @@ -1393,12 +1582,8 @@ SearchCatCacheList(CatCache *cache, */ if (cl->nkeys != nkeys) continue; - HeapKeyTest(&cl->tuple, - cache->cc_tupdesc, - nkeys, - cur_skey, - res); - if (!res) + + if (!CatalogCacheCompareTuple(cache, nkeys, cl->keys, arguments)) continue; /* @@ -1441,9 +1626,20 @@ SearchCatCacheList(CatCache *cache, PG_TRY(); { + ScanKeyData cur_skey[CATCACHE_MAXKEYS]; Relation relation; SysScanDesc scandesc; + /* + * Ok, need to make a lookup in the relation, copy the scankey and fill out + * any per-call fields. + */ + memcpy(cur_skey, cache->cc_skey, sizeof(ScanKeyData) * cache->cc_nkeys); + cur_skey[0].sk_argument = v1; + cur_skey[1].sk_argument = v2; + cur_skey[2].sk_argument = v3; + cur_skey[3].sk_argument = v4; + relation = heap_open(cache->cc_reloid, AccessShareLock); scandesc = systable_beginscan(relation, @@ -1467,7 +1663,7 @@ SearchCatCacheList(CatCache *cache, * See if there's an entry for this tuple already. */ ct = NULL; - hashValue = CatalogCacheComputeTupleHashValue(cache, ntp); + hashValue = CatalogCacheComputeTupleHashValue(cache, cache->cc_nkeys, ntp); hashIndex = HASH_INDEX(hashValue, cache->cc_nbuckets); bucket = &cache->cc_bucket[hashIndex]; @@ -1498,7 +1694,7 @@ SearchCatCacheList(CatCache *cache, if (!found) { /* We didn't find a usable entry, so make a new one */ - ct = CatalogCacheCreateEntry(cache, ntp, + ct = CatalogCacheCreateEntry(cache, ntp, arguments, hashValue, hashIndex, false); } @@ -1513,18 +1709,16 @@ SearchCatCacheList(CatCache *cache, heap_close(relation, AccessShareLock); - /* - * Now we can build the CatCList entry. First we need a dummy tuple - * containing the key values... - */ - ntp = build_dummy_tuple(cache, nkeys, cur_skey); + /* Now we can build the CatCList entry. */ oldcxt = MemoryContextSwitchTo(CacheMemoryContext); nmembers = list_length(ctlist); cl = (CatCList *) palloc(offsetof(CatCList, members) + nmembers * sizeof(CatCTup *)); - heap_copytuple_with_tuple(ntp, &cl->tuple); + + /* Extract key values */ + CatCacheCopyKeys(cache->cc_tupdesc, nkeys, cache->cc_keyno, + arguments, cl->keys); MemoryContextSwitchTo(oldcxt); - heap_freetuple(ntp); /* * We are now past the last thing that could trigger an elog before we @@ -1621,35 +1815,82 @@ ReleaseCatCacheList(CatCList *list) * supplied data into it. The new entry initially has refcount 0. */ static CatCTup * -CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp, - uint32 hashValue, Index hashIndex, bool negative) +CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp, Datum *arguments, + uint32 hashValue, Index hashIndex, + bool negative) { CatCTup *ct; HeapTuple dtp; MemoryContext oldcxt; + int i; /* - * If there are any out-of-line toasted fields in the tuple, expand them - * in-line. This saves cycles during later use of the catcache entry, and - * also protects us against the possibility of the toast tuples being - * freed before we attempt to fetch them, in case of something using a - * slightly stale catcache entry. */ - if (HeapTupleHasExternal(ntp)) - dtp = toast_flatten_tuple(ntp, cache->cc_tupdesc); + + /* negative entries have no tuple associated */ + if (ntp) + { + Assert(!negative); + + /* + * If there are any out-of-line toasted fields in the tuple, expand them + * in-line. This saves cycles during later use of the catcache entry, and + * also protects us against the possibility of the toast tuples being + * freed before we attempt to fetch them, in case of something using a + * slightly stale catcache entry. + */ + if (HeapTupleHasExternal(ntp)) + dtp = toast_flatten_tuple(ntp, cache->cc_tupdesc); + else + dtp = ntp; + + /* Allocate memory for CatCTup and the cached tuple in one go */ + oldcxt = MemoryContextSwitchTo(CacheMemoryContext); + + ct = (CatCTup *) palloc(sizeof(CatCTup) + + MAXIMUM_ALIGNOF + dtp->t_len); + ct->tuple.t_len = dtp->t_len; + ct->tuple.t_self = dtp->t_self; + ct->tuple.t_tableOid = dtp->t_tableOid; + ct->tuple.t_data = (HeapTupleHeader) + MAXALIGN(((char *) ct) + sizeof(CatCTup)); + /* copy tuple contents */ + memcpy((char *) ct->tuple.t_data, + (const char *) dtp->t_data, + dtp->t_len); + MemoryContextSwitchTo(oldcxt); + + if (dtp != ntp) + heap_freetuple(dtp); + + /* extract keys - they'll point into the tuple if not by-value */ + for (i = 0; i < cache->cc_nkeys; i++) + { + Datum atp; + bool isnull; + + atp = heap_getattr(&ct->tuple, + cache->cc_keyno[i], + cache->cc_tupdesc, + &isnull); + Assert(!isnull); + ct->keys[i] = atp; + } + } else - dtp = ntp; + { + Assert(negative); + oldcxt = MemoryContextSwitchTo(CacheMemoryContext); + ct = (CatCTup *) palloc(sizeof(CatCTup)); - /* - * Allocate CatCTup header in cache memory, and copy the tuple there too. - */ - oldcxt = MemoryContextSwitchTo(CacheMemoryContext); - ct = (CatCTup *) palloc(sizeof(CatCTup)); - heap_copytuple_with_tuple(dtp, &ct->tuple); - MemoryContextSwitchTo(oldcxt); - - if (dtp != ntp) - heap_freetuple(dtp); + /* + * Store keys - they'll point into separately allocated memory if not + * by-value. + */ + CatCacheCopyKeys(cache->cc_tupdesc, cache->cc_nkeys, cache->cc_keyno, + arguments, ct->keys); + MemoryContextSwitchTo(oldcxt); + } /* * Finish initializing the CatCTup header, and add it to the cache's @@ -1679,71 +1920,66 @@ CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp, } /* - * build_dummy_tuple - * Generate a palloc'd HeapTuple that contains the specified key - * columns, and NULLs for other columns. - * - * This is used to store the keys for negative cache entries and CatCList - * entries, which don't have real tuples associated with them. + * Helper routine that frees keys stored in the keys array. */ -static HeapTuple -build_dummy_tuple(CatCache *cache, int nkeys, ScanKey skeys) +static void +CatCacheFreeKeys(TupleDesc tupdesc, int nkeys, int *attnos, Datum *keys) { - HeapTuple ntp; - TupleDesc tupDesc = cache->cc_tupdesc; - Datum *values; - bool *nulls; - Oid tupOid = InvalidOid; - NameData tempNames[4]; - int i; + int i; - values = (Datum *) palloc(tupDesc->natts * sizeof(Datum)); - nulls = (bool *) palloc(tupDesc->natts * sizeof(bool)); + for (i = 0; i <nkeys; i++) + { + int attnum = attnos[i]; + Form_pg_attribute att; - memset(values, 0, tupDesc->natts * sizeof(Datum)); - memset(nulls, true, tupDesc->natts * sizeof(bool)); + /* only valid system attribute is the oid, which is by value */ + if (attnum == ObjectIdAttributeNumber) + continue; + Assert(attnum > 0); + + att = TupleDescAttr(tupdesc, attnum - 1); + + if (!att->attbyval) + pfree(DatumGetPointer(keys[i])); + } +} + +/* + * Helper routine that copies the keys in the srckeys array into the dstkeys + * one, guaranteeing that the datums are fully allocated in the current memory + * context. + */ +static void +CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, + Datum *srckeys, Datum *dstkeys) +{ + int i; + + /* + * XXX: memory and lookup performance could possibly be improved by + * storing all keys in one allocation. + */ for (i = 0; i < nkeys; i++) { - int attindex = cache->cc_key[i]; - Datum keyval = skeys[i].sk_argument; + int attnum = attnos[i]; - if (attindex > 0) + if (attnum != ObjectIdAttributeNumber) { - /* - * Here we must be careful in case the caller passed a C string - * where a NAME is wanted: convert the given argument to a - * correctly padded NAME. Otherwise the memcpy() done in - * heap_form_tuple could fall off the end of memory. - */ - if (cache->cc_isname[i]) - { - Name newval = &tempNames[i]; + Form_pg_attribute att = TupleDescAttr(tupdesc, attnum - 1); - namestrcpy(newval, DatumGetCString(keyval)); - keyval = NameGetDatum(newval); - } - values[attindex - 1] = keyval; - nulls[attindex - 1] = false; + dstkeys[i] = datumCopy(srckeys[i], + att->attbyval, + att->attlen); } else { - Assert(attindex == ObjectIdAttributeNumber); - tupOid = DatumGetObjectId(keyval); + dstkeys[i] = srckeys[i]; } } - ntp = heap_form_tuple(tupDesc, values, nulls); - if (tupOid != InvalidOid) - HeapTupleSetOid(ntp, tupOid); - - pfree(values); - pfree(nulls); - - return ntp; } - /* * PrepareToInvalidateCacheTuple() * @@ -1820,7 +2056,7 @@ PrepareToInvalidateCacheTuple(Relation relation, if (ccp->cc_tupdesc == NULL) CatalogCacheInitializeCache(ccp); - hashvalue = CatalogCacheComputeTupleHashValue(ccp, tuple); + hashvalue = CatalogCacheComputeTupleHashValue(ccp, ccp->cc_nkeys, tuple); dbid = ccp->cc_relisshared ? (Oid) 0 : MyDatabaseId; (*function) (ccp->id, hashvalue, dbid); @@ -1829,7 +2065,7 @@ PrepareToInvalidateCacheTuple(Relation relation, { uint32 newhashvalue; - newhashvalue = CatalogCacheComputeTupleHashValue(ccp, newtuple); + newhashvalue = CatalogCacheComputeTupleHashValue(ccp, ccp->cc_nkeys, newtuple); if (newhashvalue != hashvalue) (*function) (ccp->id, newhashvalue, dbid); diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c index fcbb683a99..888edbb325 100644 --- a/src/backend/utils/cache/syscache.c +++ b/src/backend/utils/cache/syscache.c @@ -1102,13 +1102,56 @@ SearchSysCache(int cacheId, Datum key3, Datum key4) { - if (cacheId < 0 || cacheId >= SysCacheSize || - !PointerIsValid(SysCache[cacheId])) - elog(ERROR, "invalid cache ID: %d", cacheId); + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); return SearchCatCache(SysCache[cacheId], key1, key2, key3, key4); } +HeapTuple +SearchSysCache1(int cacheId, + Datum key1) +{ + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); + Assert(SysCache[cacheId]->cc_nkeys == 1); + + return SearchCatCache1(SysCache[cacheId], key1); +} + +HeapTuple +SearchSysCache2(int cacheId, + Datum key1, Datum key2) +{ + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); + Assert(SysCache[cacheId]->cc_nkeys == 2); + + return SearchCatCache2(SysCache[cacheId], key1, key2); +} + +HeapTuple +SearchSysCache3(int cacheId, + Datum key1, Datum key2, Datum key3) +{ + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); + Assert(SysCache[cacheId]->cc_nkeys == 3); + + return SearchCatCache3(SysCache[cacheId], key1, key2, key3); +} + +HeapTuple +SearchSysCache4(int cacheId, + Datum key1, Datum key2, Datum key3, Datum key4) +{ + Assert(cacheId >= 0 && cacheId < SysCacheSize && + PointerIsValid(SysCache[cacheId])); + Assert(SysCache[cacheId]->cc_nkeys == 4); + + return SearchCatCache4(SysCache[cacheId], key1, key2, key3, key4); +} + /* * ReleaseSysCache * Release previously grabbed reference count on a tuple diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h index 200a3022e7..dc8375f829 100644 --- a/src/include/utils/catcache.h +++ b/src/include/utils/catcache.h @@ -34,26 +34,31 @@ #define CATCACHE_MAXKEYS 4 + +/* function computing a datum's hash */ +typedef uint32 (*CCHashFN) (Datum datum); +/* function computing equality of two datums */ +typedef bool (*CCFastEqualFN) (Datum a, Datum b); + typedef struct catcache { int id; /* cache identifier --- see syscache.h */ - slist_node cc_next; /* list link */ + int cc_nbuckets; /* # of hash buckets in this cache */ + TupleDesc cc_tupdesc; /* tuple descriptor (copied from reldesc) */ + dlist_head *cc_bucket; /* hash buckets */ + CCHashFN cc_hashfunc[CATCACHE_MAXKEYS]; /* hash function for each key */ + CCFastEqualFN cc_fastequal[CATCACHE_MAXKEYS]; /* fast equal function for each key */ + int cc_keyno[CATCACHE_MAXKEYS]; /* AttrNumber of each key */ + dlist_head cc_lists; /* list of CatCList structs */ + int cc_ntup; /* # of tuples currently in this cache */ + int cc_nkeys; /* # of keys (1..CATCACHE_MAXKEYS) */ const char *cc_relname; /* name of relation the tuples come from */ Oid cc_reloid; /* OID of relation the tuples come from */ Oid cc_indexoid; /* OID of index matching cache keys */ bool cc_relisshared; /* is relation shared across databases? */ - TupleDesc cc_tupdesc; /* tuple descriptor (copied from reldesc) */ - int cc_ntup; /* # of tuples currently in this cache */ - int cc_nbuckets; /* # of hash buckets in this cache */ - int cc_nkeys; /* # of keys (1..CATCACHE_MAXKEYS) */ - int cc_key[CATCACHE_MAXKEYS]; /* AttrNumber of each key */ - PGFunction cc_hashfunc[CATCACHE_MAXKEYS]; /* hash function for each key */ + slist_node cc_next; /* list link */ ScanKeyData cc_skey[CATCACHE_MAXKEYS]; /* precomputed key info for heap * scans */ - bool cc_isname[CATCACHE_MAXKEYS]; /* flag "name" key columns */ - dlist_head cc_lists; /* list of CatCList structs */ - dlist_head *cc_bucket; /* hash buckets */ - /* * Keep these at the end, so that compiling catcache.c with CATCACHE_STATS * doesn't break ABI for other modules @@ -79,7 +84,14 @@ typedef struct catctup { int ct_magic; /* for identifying CatCTup entries */ #define CT_MAGIC 0x57261502 - CatCache *my_cache; /* link to owning catcache */ + + uint32 hash_value; /* hash value for this tuple's keys */ + + /* + * Lookup keys for the entry. By-reference datums point into the tuple for + * positive cache entries, and are separately allocated for negative ones. + */ + Datum keys[CATCACHE_MAXKEYS]; /* * Each tuple in a cache is a member of a dlist that stores the elements @@ -88,15 +100,6 @@ typedef struct catctup */ dlist_node cache_elem; /* list member of per-bucket list */ - /* - * The tuple may also be a member of at most one CatCList. (If a single - * catcache is list-searched with varying numbers of keys, we may have to - * make multiple entries for the same tuple because of this restriction. - * Currently, that's not expected to be common, so we accept the potential - * inefficiency.) - */ - struct catclist *c_list; /* containing CatCList, or NULL if none */ - /* * A tuple marked "dead" must not be returned by subsequent searches. * However, it won't be physically deleted from the cache until its @@ -112,46 +115,64 @@ typedef struct catctup int refcount; /* number of active references */ bool dead; /* dead but not yet removed? */ bool negative; /* negative cache entry? */ - uint32 hash_value; /* hash value for this tuple's keys */ HeapTupleData tuple; /* tuple management header */ + + /* + * The tuple may also be a member of at most one CatCList. (If a single + * catcache is list-searched with varying numbers of keys, we may have to + * make multiple entries for the same tuple because of this restriction. + * Currently, that's not expected to be common, so we accept the potential + * inefficiency.) + */ + struct catclist *c_list; /* containing CatCList, or NULL if none */ + + CatCache *my_cache; /* link to owning catcache */ + /* properly aligned tuple data follows, unless a negative entry */ } CatCTup; +/* + * A CatCList describes the result of a partial search, ie, a search using + * only the first K key columns of an N-key cache. We store the keys used + * into the keys attribute (with other attributes NULL) to represent the + * stored key set. The CatCList object contains links to cache entries + * for all the table rows satisfying the partial key. (Note: none of + * these will be negative cache entries.) + * + * A CatCList is only a member of a per-cache list; we do not currently + * divide them into hash buckets. + * + * A list marked "dead" must not be returned by subsequent searches. + * However, it won't be physically deleted from the cache until its + * refcount goes to zero. (A list should be marked dead if any of its + * member entries are dead.) + * + * If "ordered" is true then the member tuples appear in the order of the + * cache's underlying index. This will be true in normal operation, but + * might not be true during bootstrap or recovery operations. (namespace.c + * is able to save some cycles when it is true.) + */ typedef struct catclist { int cl_magic; /* for identifying CatCList entries */ #define CL_MAGIC 0x52765103 - CatCache *my_cache; /* link to owning catcache */ + + uint32 hash_value; /* hash value for lookup keys */ + + dlist_node cache_elem; /* list member of per-catcache list */ /* - * A CatCList describes the result of a partial search, ie, a search using - * only the first K key columns of an N-key cache. We form the keys used - * into a tuple (with other attributes NULL) to represent the stored key - * set. The CatCList object contains links to cache entries for all the - * table rows satisfying the partial key. (Note: none of these will be - * negative cache entries.) - * - * A CatCList is only a member of a per-cache list; we do not currently - * divide them into hash buckets. - * - * A list marked "dead" must not be returned by subsequent searches. - * However, it won't be physically deleted from the cache until its - * refcount goes to zero. (A list should be marked dead if any of its - * member entries are dead.) - * - * If "ordered" is true then the member tuples appear in the order of the - * cache's underlying index. This will be true in normal operation, but - * might not be true during bootstrap or recovery operations. (namespace.c - * is able to save some cycles when it is true.) + * Lookup keys for the entry, with the first nkeys elements being + * valid. All by-reference are separately allocated. */ - dlist_node cache_elem; /* list member of per-catcache list */ + Datum keys[CATCACHE_MAXKEYS]; + int refcount; /* number of active references */ bool dead; /* dead but not yet removed? */ bool ordered; /* members listed in index order? */ short nkeys; /* number of lookup keys specified */ - uint32 hash_value; /* hash value for lookup keys */ - HeapTupleData tuple; /* header for tuple holding keys */ int n_members; /* number of member tuples */ + CatCache *my_cache; /* link to owning catcache */ CatCTup *members[FLEXIBLE_ARRAY_MEMBER]; /* members */ } CatCList; @@ -174,8 +195,15 @@ extern CatCache *InitCatCache(int id, Oid reloid, Oid indexoid, extern void InitCatCachePhase2(CatCache *cache, bool touch_index); extern HeapTuple SearchCatCache(CatCache *cache, - Datum v1, Datum v2, - Datum v3, Datum v4); + Datum v1, Datum v2, Datum v3, Datum v4); +extern HeapTuple SearchCatCache1(CatCache *cache, + Datum v1); +extern HeapTuple SearchCatCache2(CatCache *cache, + Datum v1, Datum v2); +extern HeapTuple SearchCatCache3(CatCache *cache, + Datum v1, Datum v2, Datum v3); +extern HeapTuple SearchCatCache4(CatCache *cache, + Datum v1, Datum v2, Datum v3, Datum v4); extern void ReleaseCatCache(HeapTuple tuple); extern uint32 GetCatCacheHashValue(CatCache *cache, diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h index 8a92ea27ac..12bda02cd7 100644 --- a/src/include/utils/syscache.h +++ b/src/include/utils/syscache.h @@ -117,6 +117,20 @@ extern void InitCatalogCachePhase2(void); extern HeapTuple SearchSysCache(int cacheId, Datum key1, Datum key2, Datum key3, Datum key4); + +/* + * The use of argument specific numbers is encouraged, they're faster, and + * insulates the caller from changes in the maximum number of keys. + */ +extern HeapTuple SearchSysCache1(int cacheId, + Datum key1); +extern HeapTuple SearchSysCache2(int cacheId, + Datum key1, Datum key2); +extern HeapTuple SearchSysCache3(int cacheId, + Datum key1, Datum key2, Datum key3); +extern HeapTuple SearchSysCache4(int cacheId, + Datum key1, Datum key2, Datum key3, Datum key4); + extern void ReleaseSysCache(HeapTuple tuple); /* convenience routines */ @@ -156,15 +170,6 @@ extern bool RelationSupportsSysCache(Oid relid); * functions is encouraged, as it insulates the caller from changes in the * maximum number of keys. */ -#define SearchSysCache1(cacheId, key1) \ - SearchSysCache(cacheId, key1, 0, 0, 0) -#define SearchSysCache2(cacheId, key1, key2) \ - SearchSysCache(cacheId, key1, key2, 0, 0) -#define SearchSysCache3(cacheId, key1, key2, key3) \ - SearchSysCache(cacheId, key1, key2, key3, 0) -#define SearchSysCache4(cacheId, key1, key2, key3, key4) \ - SearchSysCache(cacheId, key1, key2, key3, key4) - #define SearchSysCacheCopy1(cacheId, key1) \ SearchSysCacheCopy(cacheId, key1, 0, 0, 0) #define SearchSysCacheCopy2(cacheId, key1, key2) \ -- 2.14.1.536.g6867272d5b.dirty --jbfc36cenh7fvrgl Content-Type: application/x-sql Content-Disposition: attachment; filename="pgbench-many-cols.sql" Content-Transfer-Encoding: quoted-printable \set aid1 random(1, 10000)=0A= SELECT aid, field000000000000000000000000000000001,field0000000000000000000= 00000000000002,field000000000000000000000000000000003,field0000000000000000= 00000000000000004,field000000000000000000000000000000005,field0000000000000= 00000000000000000006,field000000000000000000000000000000007,field0000000000= 00000000000000000000008,field000000000000000000000000000000009,field0000000= 000000000000000000000000010,field0000000000000000000000000000000011,field00= 00000000000000000000000000000012,field0000000000000000000000000000000013,fi= eld0000000000000000000000000000000014,field00000000000000000000000000000000= 15,field0000000000000000000000000000000016,field000000000000000000000000000= 0000017,field0000000000000000000000000000000018,field0000000000000000000000= 000000000019,field0000000000000000000000000000000020,field00000000000000000= 00000000000000021,field0000000000000000000000000000000022,field000000000000= 0000000000000000000023,field0000000000000000000000000000000024,field0000000= 000000000000000000000000025,field0000000000000000000000000000000026,field00= 00000000000000000000000000000027,field0000000000000000000000000000000028,fi= eld0000000000000000000000000000000029,field00000000000000000000000000000000= 30,field0000000000000000000000000000000031,field000000000000000000000000000= 0000032,field0000000000000000000000000000000033,field0000000000000000000000= 000000000034,field0000000000000000000000000000000035,field00000000000000000= 00000000000000036,field0000000000000000000000000000000037,field000000000000= 0000000000000000000038,field0000000000000000000000000000000039,field0000000= 000000000000000000000000040,field0000000000000000000000000000000041,field00= 00000000000000000000000000000042,field0000000000000000000000000000000043,fi= eld0000000000000000000000000000000044,field00000000000000000000000000000000= 45,field0000000000000000000000000000000046,field000000000000000000000000000= 0000047,field0000000000000000000000000000000048,field0000000000000000000000= 000000000049,field0000000000000000000000000000000050,field00000000000000000= 00000000000000051,field0000000000000000000000000000000052,field000000000000= 0000000000000000000053,field0000000000000000000000000000000054,field0000000= 000000000000000000000000055,field0000000000000000000000000000000056,field00= 00000000000000000000000000000057,field0000000000000000000000000000000058,fi= eld0000000000000000000000000000000059,field00000000000000000000000000000000= 60,field0000000000000000000000000000000061,field000000000000000000000000000= 0000062,field0000000000000000000000000000000063,field0000000000000000000000= 000000000064,field0000000000000000000000000000000065,field00000000000000000= 00000000000000066,field0000000000000000000000000000000067,field000000000000= 0000000000000000000068,field0000000000000000000000000000000069,field0000000= 000000000000000000000000070,field0000000000000000000000000000000071,field00= 00000000000000000000000000000072,field0000000000000000000000000000000073,fi= eld0000000000000000000000000000000074,field00000000000000000000000000000000= 75,field0000000000000000000000000000000076,field000000000000000000000000000= 0000077,field0000000000000000000000000000000078,field0000000000000000000000= 000000000079,field0000000000000000000000000000000080,field00000000000000000= 00000000000000081,field0000000000000000000000000000000082,field000000000000= 0000000000000000000083,field0000000000000000000000000000000084,field0000000= 000000000000000000000000085,field0000000000000000000000000000000086,field00= 00000000000000000000000000000087,field0000000000000000000000000000000088,fi= eld0000000000000000000000000000000089,field00000000000000000000000000000000= 90,field0000000000000000000000000000000091,field000000000000000000000000000= 0000092,field0000000000000000000000000000000093,field0000000000000000000000= 000000000094,field0000000000000000000000000000000095,field00000000000000000= 00000000000000096,field0000000000000000000000000000000097,field000000000000= 0000000000000000000098,field0000000000000000000000000000000099,field0000000= 0000000000000000000000000100 FROM data_import WHERE aid =3D :aid1;=0A= --jbfc36cenh7fvrgl Content-Type: application/x-sql Content-Disposition: attachment; filename="create_many_cols.sql" Content-Transfer-Encoding: quoted-printable DROP TABLE IF EXISTS data_import;=0A= CREATE TABLE data_import(aid int PRIMARY KEY, field000000000000000000000000= 000000001 int,field000000000000000000000000000000002 int,field0000000000000= 00000000000000000003 int,field000000000000000000000000000000004 int,field00= 0000000000000000000000000000005 int,field000000000000000000000000000000006 = int,field000000000000000000000000000000007 int,field00000000000000000000000= 0000000008 int,field000000000000000000000000000000009 int,field000000000000= 0000000000000000000010 int,field0000000000000000000000000000000011 int,fiel= d0000000000000000000000000000000012 int,field000000000000000000000000000000= 0013 int,field0000000000000000000000000000000014 int,field00000000000000000= 00000000000000015 int,field0000000000000000000000000000000016 int,field0000= 000000000000000000000000000017 int,field0000000000000000000000000000000018 = int,field0000000000000000000000000000000019 int,field0000000000000000000000= 000000000020 int,field0000000000000000000000000000000021 int,field000000000= 0000000000000000000000022 int,field0000000000000000000000000000000023 int,f= ield0000000000000000000000000000000024 int,field000000000000000000000000000= 0000025 int,field0000000000000000000000000000000026 int,field00000000000000= 00000000000000000027 int,field0000000000000000000000000000000028 int,field0= 000000000000000000000000000000029 int,field00000000000000000000000000000000= 30 int,field0000000000000000000000000000000031 int,field0000000000000000000= 000000000000032 int,field0000000000000000000000000000000033 int,field000000= 0000000000000000000000000034 int,field0000000000000000000000000000000035 in= t,field0000000000000000000000000000000036 int,field000000000000000000000000= 0000000037 int,field0000000000000000000000000000000038 int,field00000000000= 00000000000000000000039 int,field0000000000000000000000000000000040 int,fie= ld0000000000000000000000000000000041 int,field00000000000000000000000000000= 00042 int,field0000000000000000000000000000000043 int,field0000000000000000= 000000000000000044 int,field0000000000000000000000000000000045 int,field000= 0000000000000000000000000000046 int,field0000000000000000000000000000000047= int,field0000000000000000000000000000000048 int,field000000000000000000000= 0000000000049 int,field0000000000000000000000000000000050 int,field00000000= 00000000000000000000000051 int,field0000000000000000000000000000000052 int,= field0000000000000000000000000000000053 int,field00000000000000000000000000= 00000054 int,field0000000000000000000000000000000055 int,field0000000000000= 000000000000000000056 int,field0000000000000000000000000000000057 int,field= 0000000000000000000000000000000058 int,field0000000000000000000000000000000= 059 int,field0000000000000000000000000000000060 int,field000000000000000000= 0000000000000061 int,field0000000000000000000000000000000062 int,field00000= 00000000000000000000000000063 int,field0000000000000000000000000000000064 i= nt,field0000000000000000000000000000000065 int,field00000000000000000000000= 00000000066 int,field0000000000000000000000000000000067 int,field0000000000= 000000000000000000000068 int,field0000000000000000000000000000000069 int,fi= eld0000000000000000000000000000000070 int,field0000000000000000000000000000= 000071 int,field0000000000000000000000000000000072 int,field000000000000000= 0000000000000000073 int,field0000000000000000000000000000000074 int,field00= 00000000000000000000000000000075 int,field000000000000000000000000000000007= 6 int,field0000000000000000000000000000000077 int,field00000000000000000000= 00000000000078 int,field0000000000000000000000000000000079 int,field0000000= 000000000000000000000000080 int,field0000000000000000000000000000000081 int= ,field0000000000000000000000000000000082 int,field0000000000000000000000000= 000000083 int,field0000000000000000000000000000000084 int,field000000000000= 0000000000000000000085 int,field0000000000000000000000000000000086 int,fiel= d0000000000000000000000000000000087 int,field000000000000000000000000000000= 0088 int,field0000000000000000000000000000000089 int,field00000000000000000= 00000000000000090 int,field0000000000000000000000000000000091 int,field0000= 000000000000000000000000000092 int,field0000000000000000000000000000000093 = int,field0000000000000000000000000000000094 int,field0000000000000000000000= 000000000095 int,field0000000000000000000000000000000096 int,field000000000= 0000000000000000000000097 int,field0000000000000000000000000000000098 int,f= ield0000000000000000000000000000000099 int,field000000000000000000000000000= 00000100 int,field00000000000000000000000000000000101 int,field000000000000= 00000000000000000000102 int,field00000000000000000000000000000000103 int,fi= eld00000000000000000000000000000000104 int,field000000000000000000000000000= 00000105 int,field00000000000000000000000000000000106 int,field000000000000= 00000000000000000000107 int,field00000000000000000000000000000000108 int,fi= eld00000000000000000000000000000000109 int,field000000000000000000000000000= 00000110 int,field00000000000000000000000000000000111 int,field000000000000= 00000000000000000000112 int,field00000000000000000000000000000000113 int,fi= eld00000000000000000000000000000000114 int,field000000000000000000000000000= 00000115 int,field00000000000000000000000000000000116 int,field000000000000= 00000000000000000000117 int,field00000000000000000000000000000000118 int,fi= eld00000000000000000000000000000000119 int,field000000000000000000000000000= 00000120 int,field00000000000000000000000000000000121 int,field000000000000= 00000000000000000000122 int,field00000000000000000000000000000000123 int,fi= eld00000000000000000000000000000000124 int,field000000000000000000000000000= 00000125 int,field00000000000000000000000000000000126 int,field000000000000= 00000000000000000000127 int,field00000000000000000000000000000000128 int,fi= eld00000000000000000000000000000000129 int,field000000000000000000000000000= 00000130 int,field00000000000000000000000000000000131 int,field000000000000= 00000000000000000000132 int,field00000000000000000000000000000000133 int,fi= eld00000000000000000000000000000000134 int,field000000000000000000000000000= 00000135 int,field00000000000000000000000000000000136 int,field000000000000= 00000000000000000000137 int,field00000000000000000000000000000000138 int,fi= eld00000000000000000000000000000000139 int,field000000000000000000000000000= 00000140 int,field00000000000000000000000000000000141 int,field000000000000= 00000000000000000000142 int,field00000000000000000000000000000000143 int,fi= eld00000000000000000000000000000000144 int,field000000000000000000000000000= 00000145 int,field00000000000000000000000000000000146 int,field000000000000= 00000000000000000000147 int,field00000000000000000000000000000000148 int,fi= eld00000000000000000000000000000000149 int,field000000000000000000000000000= 00000150 int);=0A= insert into data_import select generate_series(1, 10000);=0A= vacuum freeze analyze data_import;=0A= --jbfc36cenh7fvrgl Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 8bit MIME-Version: 1.0 -- Sent via pgsql-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers --jbfc36cenh7fvrgl-- ^ permalink raw reply [nested|flat] 6+ messages in thread
* [PATCH v47 3/7] Make archiver process an auxiliary process @ 2020-03-13 07:59 Kyotaro Horiguchi <[email protected]> 0 siblings, 0 replies; 6+ messages in thread From: Kyotaro Horiguchi @ 2020-03-13 07:59 UTC (permalink / raw) This is a preliminary patch for shared-memory based stats collector. Archiver process must be a auxiliary process since it uses shared memory after stats data was moved into shared-memory. Make the process an auxiliary process in order to make it work. --- src/backend/access/transam/xlogarchive.c | 6 +- src/backend/bootstrap/bootstrap.c | 22 ++-- src/backend/postmaster/pgarch.c | 130 +++-------------------- src/backend/postmaster/postmaster.c | 50 +++++---- src/backend/storage/lmgr/proc.c | 1 + src/include/access/xlog.h | 3 + src/include/access/xlogarchive.h | 1 + src/include/miscadmin.h | 2 + src/include/postmaster/pgarch.h | 4 +- src/include/storage/pmsignal.h | 1 - src/include/storage/proc.h | 3 + 11 files changed, 69 insertions(+), 154 deletions(-) diff --git a/src/backend/access/transam/xlogarchive.c b/src/backend/access/transam/xlogarchive.c index 1c5a4f8b5a..d01859bde5 100644 --- a/src/backend/access/transam/xlogarchive.c +++ b/src/backend/access/transam/xlogarchive.c @@ -29,7 +29,9 @@ #include "storage/fd.h" #include "storage/ipc.h" #include "storage/lwlock.h" +#include "storage/latch.h" #include "storage/pmsignal.h" +#include "storage/proc.h" /* * Attempt to retrieve the specified file from off-line archival storage. @@ -490,8 +492,8 @@ XLogArchiveNotify(const char *xlog) } /* Notify archiver that it's got something to do */ - if (IsUnderPostmaster) - SendPostmasterSignal(PMSIGNAL_WAKEN_ARCHIVER); + if (IsUnderPostmaster && ProcGlobal->archiverLatch) + SetLatch(ProcGlobal->archiverLatch); } /* diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c index 6f615e6622..41da0c5059 100644 --- a/src/backend/bootstrap/bootstrap.c +++ b/src/backend/bootstrap/bootstrap.c @@ -317,6 +317,9 @@ AuxiliaryProcessMain(int argc, char *argv[]) case StartupProcess: MyBackendType = B_STARTUP; break; + case ArchiverProcess: + MyBackendType = B_ARCHIVER; + break; case BgWriterProcess: MyBackendType = B_BG_WRITER; break; @@ -437,30 +440,29 @@ AuxiliaryProcessMain(int argc, char *argv[]) proc_exit(1); /* should never return */ case StartupProcess: - /* don't set signals, startup process has its own agenda */ StartupProcessMain(); - proc_exit(1); /* should never return */ + proc_exit(1); + + case ArchiverProcess: + PgArchiverMain(); + proc_exit(1); case BgWriterProcess: - /* don't set signals, bgwriter has its own agenda */ BackgroundWriterMain(); - proc_exit(1); /* should never return */ + proc_exit(1); case CheckpointerProcess: - /* don't set signals, checkpointer has its own agenda */ CheckpointerMain(); - proc_exit(1); /* should never return */ + proc_exit(1); case WalWriterProcess: - /* don't set signals, walwriter has its own agenda */ InitXLOGAccess(); WalWriterMain(); - proc_exit(1); /* should never return */ + proc_exit(1); case WalReceiverProcess: - /* don't set signals, walreceiver has its own agenda */ WalReceiverMain(); - proc_exit(1); /* should never return */ + proc_exit(1); default: elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType); diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c index edec311f12..9a2e21bf86 100644 --- a/src/backend/postmaster/pgarch.c +++ b/src/backend/postmaster/pgarch.c @@ -48,6 +48,7 @@ #include "storage/latch.h" #include "storage/pg_shmem.h" #include "storage/pmsignal.h" +#include "storage/procsignal.h" #include "utils/guc.h" #include "utils/ps_status.h" @@ -78,13 +79,11 @@ * Local data * ---------- */ -static time_t last_pgarch_start_time; static time_t last_sigterm_time = 0; /* * Flags set by interrupt handlers for later service in the main loop. */ -static volatile sig_atomic_t wakened = false; static volatile sig_atomic_t ready_to_stop = false; /* ---------- @@ -95,8 +94,6 @@ static volatile sig_atomic_t ready_to_stop = false; static pid_t pgarch_forkexec(void); #endif -NON_EXEC_STATIC void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn(); -static void pgarch_waken(SIGNAL_ARGS); static void pgarch_waken_stop(SIGNAL_ARGS); static void pgarch_MainLoop(void); static void pgarch_ArchiverCopyLoop(void); @@ -110,75 +107,6 @@ static void pgarch_archiveDone(char *xlog); * ------------------------------------------------------------ */ -/* - * pgarch_start - * - * Called from postmaster at startup or after an existing archiver - * died. Attempt to fire up a fresh archiver process. - * - * Returns PID of child process, or 0 if fail. - * - * Note: if fail, we will be called again from the postmaster main loop. - */ -int -pgarch_start(void) -{ - time_t curtime; - pid_t pgArchPid; - - /* - * Do nothing if no archiver needed - */ - if (!XLogArchivingActive()) - return 0; - - /* - * Do nothing if too soon since last archiver start. This is a safety - * valve to protect against continuous respawn attempts if the archiver is - * dying immediately at launch. Note that since we will be re-called from - * the postmaster main loop, we will get another chance later. - */ - curtime = time(NULL); - if ((unsigned int) (curtime - last_pgarch_start_time) < - (unsigned int) PGARCH_RESTART_INTERVAL) - return 0; - last_pgarch_start_time = curtime; - -#ifdef EXEC_BACKEND - switch ((pgArchPid = pgarch_forkexec())) -#else - switch ((pgArchPid = fork_process())) -#endif - { - case -1: - ereport(LOG, - (errmsg("could not fork archiver: %m"))); - return 0; - -#ifndef EXEC_BACKEND - case 0: - /* in postmaster child ... */ - InitPostmasterChild(); - - /* Close the postmaster's sockets */ - ClosePostmasterPorts(false); - - /* Drop our connection to postmaster's shared memory, as well */ - dsm_detach_all(); - PGSharedMemoryDetach(); - - PgArchiverMain(0, NULL); - break; -#endif - - default: - return (int) pgArchPid; - } - - /* shouldn't get here */ - return 0; -} - /* ------------------------------------------------------------ * Local functions called by archiver follow * ------------------------------------------------------------ @@ -212,14 +140,9 @@ pgarch_forkexec(void) #endif /* EXEC_BACKEND */ -/* - * PgArchiverMain - * - * The argc/argv parameters are valid only in EXEC_BACKEND case. However, - * since we don't use 'em, it hardly matters... - */ -NON_EXEC_STATIC void -PgArchiverMain(int argc, char *argv[]) +/* Main entry point for archiver process */ +void +PgArchiverMain(void) { /* * Ignore all signals usually bound to some action in the postmaster, @@ -231,33 +154,26 @@ PgArchiverMain(int argc, char *argv[]) /* SIGQUIT handler was already set up by InitPostmasterChild */ pqsignal(SIGALRM, SIG_IGN); pqsignal(SIGPIPE, SIG_IGN); - pqsignal(SIGUSR1, pgarch_waken); + pqsignal(SIGUSR1, procsignal_sigusr1_handler); pqsignal(SIGUSR2, pgarch_waken_stop); + /* Reset some signals that are accepted by postmaster but not here */ pqsignal(SIGCHLD, SIG_DFL); + + /* Unblock signals (they were blocked when the postmaster forked us) */ PG_SETMASK(&UnBlockSig); - MyBackendType = B_ARCHIVER; - init_ps_display(NULL); + /* + * Advertise our latch that backends can use to wake us up while we're + * sleeping. + */ + ProcGlobal->archiverLatch = &MyProc->procLatch; pgarch_MainLoop(); exit(0); } -/* SIGUSR1 signal handler for archiver process */ -static void -pgarch_waken(SIGNAL_ARGS) -{ - int save_errno = errno; - - /* set flag that there is work to be done */ - wakened = true; - SetLatch(MyLatch); - - errno = save_errno; -} - /* SIGUSR2 signal handler for archiver process */ static void pgarch_waken_stop(SIGNAL_ARGS) @@ -282,14 +198,6 @@ pgarch_MainLoop(void) pg_time_t last_copy_time = 0; bool time_to_stop; - /* - * We run the copy loop immediately upon entry, in case there are - * unarchived files left over from a previous database run (or maybe the - * archiver died unexpectedly). After that we wait for a signal or - * timeout before doing more. - */ - wakened = true; - /* * There shouldn't be anything for the archiver to do except to wait for a * signal ... however, the archiver exists to protect our data, so she @@ -328,12 +236,8 @@ pgarch_MainLoop(void) } /* Do what we're here for */ - if (wakened || time_to_stop) - { - wakened = false; - pgarch_ArchiverCopyLoop(); - last_copy_time = time(NULL); - } + pgarch_ArchiverCopyLoop(); + last_copy_time = time(NULL); /* * Sleep until a signal is received, or until a poll is forced by @@ -354,13 +258,9 @@ pgarch_MainLoop(void) WL_LATCH_SET | WL_TIMEOUT | WL_POSTMASTER_DEATH, timeout * 1000L, WAIT_EVENT_ARCHIVER_MAIN); - if (rc & WL_TIMEOUT) - wakened = true; if (rc & WL_POSTMASTER_DEATH) time_to_stop = true; } - else - wakened = true; } /* diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c index 7de27ee4e0..af91c313e2 100644 --- a/src/backend/postmaster/postmaster.c +++ b/src/backend/postmaster/postmaster.c @@ -548,6 +548,7 @@ static void ShmemBackendArrayRemove(Backend *bn); #endif /* EXEC_BACKEND */ #define StartupDataBase() StartChildProcess(StartupProcess) +#define StartArchiver() StartChildProcess(ArchiverProcess) #define StartBackgroundWriter() StartChildProcess(BgWriterProcess) #define StartCheckpointer() StartChildProcess(CheckpointerProcess) #define StartWalWriter() StartChildProcess(WalWriterProcess) @@ -1788,7 +1789,7 @@ ServerLoop(void) /* If we have lost the archiver, try to start a new one. */ if (PgArchPID == 0 && PgArchStartupAllowed()) - PgArchPID = pgarch_start(); + PgArchPID = StartArchiver(); /* If we need to signal the autovacuum launcher, do so now */ if (avlauncher_needs_signal) @@ -3027,7 +3028,7 @@ reaper(SIGNAL_ARGS) if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0) AutoVacPID = StartAutoVacLauncher(); if (PgArchStartupAllowed() && PgArchPID == 0) - PgArchPID = pgarch_start(); + PgArchPID = StartArchiver(); if (PgStatPID == 0) PgStatPID = pgstat_start(); @@ -3162,20 +3163,16 @@ reaper(SIGNAL_ARGS) } /* - * Was it the archiver? If so, just try to start a new one; no need - * to force reset of the rest of the system. (If fail, we'll try - * again in future cycles of the main loop.). Unless we were waiting - * for it to shut down; don't restart it in that case, and - * PostmasterStateMachine() will advance to the next shutdown step. + * Was it the archiver? Normal exit can be ignored; we'll start a new + * one at the next iteration of the postmaster's main loop, if + * necessary. Any other exit condition is treated as a crash. */ if (pid == PgArchPID) { PgArchPID = 0; if (!EXIT_STATUS_0(exitstatus)) - LogChildExit(LOG, _("archiver process"), - pid, exitstatus); - if (PgArchStartupAllowed()) - PgArchPID = pgarch_start(); + HandleChildCrash(pid, exitstatus, + _("archiver process")); continue; } @@ -3423,7 +3420,7 @@ CleanupBackend(int pid, /* * HandleChildCrash -- cleanup after failed backend, bgwriter, checkpointer, - * walwriter, autovacuum, or background worker. + * walwriter, autovacuum, archiver or background worker. * * The objectives here are to clean up our local state about the child * process, and to signal all other remaining children to quickdie. @@ -3629,6 +3626,18 @@ HandleChildCrash(int pid, int exitstatus, const char *procname) signal_child(AutoVacPID, (SendStop ? SIGSTOP : SIGQUIT)); } + /* Take care of the archiver too */ + if (pid == PgArchPID) + PgArchPID = 0; + else if (PgArchPID != 0 && take_action) + { + ereport(DEBUG2, + (errmsg_internal("sending %s to process %d", + (SendStop ? "SIGSTOP" : "SIGQUIT"), + (int) PgArchPID))); + signal_child(PgArchPID, (SendStop ? SIGSTOP : SIGQUIT)); + } + /* * Force a power-cycle of the pgarch process too. (This isn't absolutely * necessary, but it seems like a good idea for robustness, and it @@ -3932,6 +3941,7 @@ PostmasterStateMachine(void) Assert(CheckpointerPID == 0); Assert(WalWriterPID == 0); Assert(AutoVacPID == 0); + Assert(PgArchPID == 0); /* syslogger is not considered here */ pmState = PM_NO_CHILDREN; } @@ -5160,7 +5170,7 @@ sigusr1_handler(SIGNAL_ARGS) */ Assert(PgArchPID == 0); if (XLogArchivingAlways()) - PgArchPID = pgarch_start(); + PgArchPID = StartArchiver(); /* * If we aren't planning to enter hot standby mode later, treat @@ -5214,16 +5224,6 @@ sigusr1_handler(SIGNAL_ARGS) if (StartWorkerNeeded || HaveCrashedWorker) maybe_start_bgworkers(); - if (CheckPostmasterSignal(PMSIGNAL_WAKEN_ARCHIVER) && - PgArchPID != 0) - { - /* - * Send SIGUSR1 to archiver process, to wake it up and begin archiving - * next WAL file. - */ - signal_child(PgArchPID, SIGUSR1); - } - /* Tell syslogger to rotate logfile if requested */ if (SysLoggerPID != 0) { @@ -5465,6 +5465,10 @@ StartChildProcess(AuxProcType type) ereport(LOG, (errmsg("could not fork startup process: %m"))); break; + case ArchiverProcess: + ereport(LOG, + (errmsg("could not fork archiver process: %m"))); + break; case BgWriterProcess: ereport(LOG, (errmsg("could not fork background writer process: %m"))); diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c index c87ffc6549..a1e51c5b99 100644 --- a/src/backend/storage/lmgr/proc.c +++ b/src/backend/storage/lmgr/proc.c @@ -182,6 +182,7 @@ InitProcGlobal(void) ProcGlobal->startupBufferPinWaitBufId = -1; ProcGlobal->walwriterLatch = NULL; ProcGlobal->checkpointerLatch = NULL; + ProcGlobal->archiverLatch = NULL; pg_atomic_init_u32(&ProcGlobal->procArrayGroupFirst, INVALID_PGPROCNO); pg_atomic_init_u32(&ProcGlobal->clogGroupFirst, INVALID_PGPROCNO); diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index 75ec1073bd..551f518cc2 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -339,6 +339,9 @@ extern XLogRecPtr GetRedoRecPtr(void); extern XLogRecPtr GetInsertRecPtr(void); extern XLogRecPtr GetFlushRecPtr(void); extern XLogRecPtr GetLastImportantRecPtr(void); +extern void XLogArchiveWakeupStart(void); +extern void XLogArchiveWakeupEnd(void); +extern void XLogArchiveWakeup(void); extern void RemovePromoteSignalFiles(void); extern bool PromoteIsTriggered(void); diff --git a/src/include/access/xlogarchive.h b/src/include/access/xlogarchive.h index 3edd1a976c..1a59181cf9 100644 --- a/src/include/access/xlogarchive.h +++ b/src/include/access/xlogarchive.h @@ -25,6 +25,7 @@ extern void ExecuteRecoveryCommand(const char *command, const char *commandName, extern void KeepFileRestoredFromArchive(const char *path, const char *xlogfname); extern void XLogArchiveNotify(const char *xlog); extern void XLogArchiveNotifySeg(XLogSegNo segno); +extern void XLogArchiveWakeup(void); extern void XLogArchiveForceDone(const char *xlog); extern bool XLogArchiveCheckDone(const char *xlog); extern bool XLogArchiveIsBusy(const char *xlog); diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h index 1bdc97e308..adb9f819bb 100644 --- a/src/include/miscadmin.h +++ b/src/include/miscadmin.h @@ -419,6 +419,7 @@ typedef enum BootstrapProcess, StartupProcess, BgWriterProcess, + ArchiverProcess, CheckpointerProcess, WalWriterProcess, WalReceiverProcess, @@ -431,6 +432,7 @@ extern AuxProcType MyAuxProcType; #define AmBootstrapProcess() (MyAuxProcType == BootstrapProcess) #define AmStartupProcess() (MyAuxProcType == StartupProcess) #define AmBackgroundWriterProcess() (MyAuxProcType == BgWriterProcess) +#define AmArchiverProcess() (MyAuxProcType == ArchiverProcess) #define AmCheckpointerProcess() (MyAuxProcType == CheckpointerProcess) #define AmWalWriterProcess() (MyAuxProcType == WalWriterProcess) #define AmWalReceiverProcess() (MyAuxProcType == WalReceiverProcess) diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h index d102a21ab7..385b002dfe 100644 --- a/src/include/postmaster/pgarch.h +++ b/src/include/postmaster/pgarch.h @@ -32,8 +32,6 @@ */ extern int pgarch_start(void); -#ifdef EXEC_BACKEND -extern void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn(); -#endif +extern void PgArchiverMain(void) pg_attribute_noreturn(); #endif /* _PGARCH_H */ diff --git a/src/include/storage/pmsignal.h b/src/include/storage/pmsignal.h index dbbed18f61..8ed4d87ae6 100644 --- a/src/include/storage/pmsignal.h +++ b/src/include/storage/pmsignal.h @@ -34,7 +34,6 @@ typedef enum { PMSIGNAL_RECOVERY_STARTED, /* recovery has started */ PMSIGNAL_BEGIN_HOT_STANDBY, /* begin Hot Standby */ - PMSIGNAL_WAKEN_ARCHIVER, /* send a NOTIFY signal to xlog archiver */ PMSIGNAL_ROTATE_LOGFILE, /* send SIGUSR1 to syslogger to rotate logfile */ PMSIGNAL_START_AUTOVAC_LAUNCHER, /* start an autovacuum launcher */ PMSIGNAL_START_AUTOVAC_WORKER, /* start an autovacuum worker */ diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h index 683ab64f76..6cdaf3753d 100644 --- a/src/include/storage/proc.h +++ b/src/include/storage/proc.h @@ -355,6 +355,9 @@ typedef struct PROC_HDR int startupProcPid; /* Buffer id of the buffer that Startup process waits for pin on, or -1 */ int startupBufferPinWaitBufId; + /* Archiver process's latch */ + Latch *archiverLatch; + /* Current shared estimate of appropriate spins_per_delay value */ } PROC_HDR; extern PGDLLIMPORT PROC_HDR *ProcGlobal; -- 2.27.0 ----Next_Part(Thu_Jan_21_12_03_48_2021_284)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v47-0004-Shared-memory-based-stats-collector.patch" ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: numeric datatype for current release not available @ 2023-07-12 15:25 Julien Rouhaud <[email protected]> 2023-07-13 04:14 ` Re: numeric datatype for current release not available Ashutosh Bapat <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: Julien Rouhaud @ 2023-07-12 15:25 UTC (permalink / raw) To: Ashutosh Bapat <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]> On Wed, 12 Jul 2023, 23:15 Ashutosh Bapat, <[email protected]> wrote: > Hi All, > https://www.postgresql.org/docs/current/datatype-numeric.html gives me > "bad gateway" error. Attached screen shot. Date/Time datatype > documentation is accessible at > https://www.postgresql.org/docs/current/datatype-datetime.html. > > Just got this while wrapping up for the day. Didn't look at what's going > wrong. > it's working here. probably a transient error, it happens from time to time. > ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: numeric datatype for current release not available 2023-07-12 15:25 Re: numeric datatype for current release not available Julien Rouhaud <[email protected]> @ 2023-07-13 04:14 ` Ashutosh Bapat <[email protected]> 0 siblings, 0 replies; 6+ messages in thread From: Ashutosh Bapat @ 2023-07-13 04:14 UTC (permalink / raw) To: Julien Rouhaud <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]> On Wed, Jul 12, 2023 at 8:55 PM Julien Rouhaud <[email protected]> wrote: > > it's working here. probably a transient error, it happens from time to time. Thanks Julien for looking into it. It's working now. -- Best Wishes, Ashutosh Bapat ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2023-07-13 04:14 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2017-09-12 01:25 [PATCH 7/8] Improve sys/catcache performance. Andres Freund <[email protected]> 2017-09-12 01:25 [PATCH 7/8] Improve sys/catcache performance. Andres Freund <[email protected]> 2017-09-12 01:25 [PATCH 3/3] Improve sys/catcache performance. Andres Freund <[email protected]> 2020-03-13 07:59 [PATCH v47 3/7] Make archiver process an auxiliary process Kyotaro Horiguchi <[email protected]> 2023-07-12 15:25 Re: numeric datatype for current release not available Julien Rouhaud <[email protected]> 2023-07-13 04:14 ` Re: numeric datatype for current release not available Ashutosh Bapat <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox