public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 2/4] Write WAL for empty nbtree index build
31+ messages / 10 participants
[nested] [flat]
* [PATCH 2/4] Write WAL for empty nbtree index build
@ 2018-10-11 01:03 Kyotaro Horiguchi <[email protected]>
0 siblings, 0 replies; 31+ messages in thread
From: Kyotaro Horiguchi @ 2018-10-11 01:03 UTC (permalink / raw)
After relation truncation indexes are also rebuild. It doesn't emit
WAL in minimal mode and if truncation happened within its creation
transaction, crash recovery leaves an empty index heap, which is
considered broken. This patch forces to emit WAL when an index_build
turns into empty nbtree index.
---
src/backend/access/nbtree/nbtsort.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index 16f5755777..2c2647b530 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -610,8 +610,14 @@ _bt_blwritepage(BTWriteState *wstate, Page page, BlockNumber blkno)
/* Ensure rd_smgr is open (could have been closed by relcache flush!) */
RelationOpenSmgr(wstate->index);
- /* XLOG stuff */
- if (wstate->btws_use_wal)
+ /* XLOG stuff
+ *
+ * Even if minimal mode, WAL is required here if truncation happened after
+ * being created in the same transaction. It is not needed otherwise but
+ * we don't bother identifying the case precisely.
+ */
+ if (wstate->btws_use_wal ||
+ (blkno == BTREE_METAPAGE && BTPageGetMeta(page)->btm_root == 0))
{
/* We use the heap NEWPAGE record type for this */
log_newpage(&wstate->index->rd_node, MAIN_FORKNUM, blkno, page, true);
@@ -1055,6 +1061,11 @@ _bt_uppershutdown(BTWriteState *wstate, BTPageState *state)
* set to point to "P_NONE"). This changes the index to the "valid" state
* by filling in a valid magic number in the metapage.
*/
+ /*
+ * If no tuple was inserted, it's possible that we are truncating a
+ * relation. We need to emit WAL for the metapage in the case. However it
+ * is not required elsewise,
+ */
metapage = (Page) palloc(BLCKSZ);
_bt_initmetapage(metapage, rootblkno, rootlevel);
_bt_blwritepage(wstate, metapage, BTREE_METAPAGE);
--
2.16.3
----Next_Part(Thu_Dec_20_17_32_25_2018_311)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="v5-0003-Add-infrastructure-to-WAL-logging-skip-feature.patch"
^ permalink raw reply [nested|flat] 31+ messages in thread
* [PATCH 2/4] Write WAL for empty nbtree index build
@ 2018-10-11 01:03 Kyotaro Horiguchi <[email protected]>
0 siblings, 0 replies; 31+ messages in thread
From: Kyotaro Horiguchi @ 2018-10-11 01:03 UTC (permalink / raw)
After relation truncation indexes are also rebuild. It doesn't emit
WAL in minimal mode and if truncation happened within its creation
transaction, crash recovery leaves an empty index heap, which is
considered broken. This patch forces to emit WAL when an index_build
turns into empty nbtree index.
---
src/backend/access/nbtree/nbtsort.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index dc398e1186..70d4380533 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -611,8 +611,14 @@ _bt_blwritepage(BTWriteState *wstate, Page page, BlockNumber blkno)
/* Ensure rd_smgr is open (could have been closed by relcache flush!) */
RelationOpenSmgr(wstate->index);
- /* XLOG stuff */
- if (wstate->btws_use_wal)
+ /* XLOG stuff
+ *
+ * Even if minimal mode, WAL is required here if truncation happened after
+ * being created in the same transaction. It is not needed otherwise but
+ * we don't bother identifying the case precisely.
+ */
+ if (wstate->btws_use_wal ||
+ (blkno == BTREE_METAPAGE && BTPageGetMeta(page)->btm_root == 0))
{
/* We use the heap NEWPAGE record type for this */
log_newpage(&wstate->index->rd_node, MAIN_FORKNUM, blkno, page, true);
@@ -1056,6 +1062,11 @@ _bt_uppershutdown(BTWriteState *wstate, BTPageState *state)
* set to point to "P_NONE"). This changes the index to the "valid" state
* by filling in a valid magic number in the metapage.
*/
+ /*
+ * If no tuple was inserted, it's possible that we are truncating a
+ * relation. We need to emit WAL for the metapage in the case. However it
+ * is not required elsewise,
+ */
metapage = (Page) palloc(BLCKSZ);
_bt_initmetapage(metapage, rootblkno, rootlevel);
_bt_blwritepage(wstate, metapage, BTREE_METAPAGE);
--
2.16.3
----Next_Part(Wed_Jan_30_10_26_34_2019_453)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="v6-0003-Add-infrastructure-to-WAL-logging-skip-feature.patch"
^ permalink raw reply [nested|flat] 31+ messages in thread
* [PATCH 2/4] Write WAL for empty nbtree index build
@ 2018-10-11 01:03 Kyotaro Horiguchi <[email protected]>
0 siblings, 0 replies; 31+ messages in thread
From: Kyotaro Horiguchi @ 2018-10-11 01:03 UTC (permalink / raw)
After relation truncation indexes are also rebuild. It doesn't emit
WAL in minimal mode and if truncation happened within its creation
transaction, crash recovery leaves an empty index heap, which is
considered broken. This patch forces to emit WAL when an index_build
turns into empty nbtree index.
---
src/backend/access/nbtree/nbtsort.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index 16f5755777..2c2647b530 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -610,8 +610,14 @@ _bt_blwritepage(BTWriteState *wstate, Page page, BlockNumber blkno)
/* Ensure rd_smgr is open (could have been closed by relcache flush!) */
RelationOpenSmgr(wstate->index);
- /* XLOG stuff */
- if (wstate->btws_use_wal)
+ /* XLOG stuff
+ *
+ * Even if minimal mode, WAL is required here if truncation happened after
+ * being created in the same transaction. It is not needed otherwise but
+ * we don't bother identifying the case precisely.
+ */
+ if (wstate->btws_use_wal ||
+ (blkno == BTREE_METAPAGE && BTPageGetMeta(page)->btm_root == 0))
{
/* We use the heap NEWPAGE record type for this */
log_newpage(&wstate->index->rd_node, MAIN_FORKNUM, blkno, page, true);
@@ -1055,6 +1061,11 @@ _bt_uppershutdown(BTWriteState *wstate, BTPageState *state)
* set to point to "P_NONE"). This changes the index to the "valid" state
* by filling in a valid magic number in the metapage.
*/
+ /*
+ * If no tuple was inserted, it's possible that we are truncating a
+ * relation. We need to emit WAL for the metapage in the case. However it
+ * is not required elsewise,
+ */
metapage = (Page) palloc(BLCKSZ);
_bt_initmetapage(metapage, rootblkno, rootlevel);
_bt_blwritepage(wstate, metapage, BTREE_METAPAGE);
--
2.16.3
----Next_Part(Wed_Nov_14_12_47_36_2018_817)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="v4-0001-TAP-test-for-copy-truncation-optimization.patch"
^ permalink raw reply [nested|flat] 31+ messages in thread
* [PATCH 3/3] Write WAL for empty nbtree index build
@ 2018-10-11 01:03 Kyotaro Horiguchi <[email protected]>
0 siblings, 0 replies; 31+ messages in thread
From: Kyotaro Horiguchi @ 2018-10-11 01:03 UTC (permalink / raw)
After relation truncation indexes are also rebuild. It doesn't emit
WAL in minimal mode and if truncation happened within its creation
transaction, crash recovery leaves an empty index heap, which is
considered broken. This patch forces to emit WAL when an index_build
turns into empty nbtree index.
---
src/backend/access/nbtree/nbtsort.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index 16f5755777..2c2647b530 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -610,8 +610,14 @@ _bt_blwritepage(BTWriteState *wstate, Page page, BlockNumber blkno)
/* Ensure rd_smgr is open (could have been closed by relcache flush!) */
RelationOpenSmgr(wstate->index);
- /* XLOG stuff */
- if (wstate->btws_use_wal)
+ /* XLOG stuff
+ *
+ * Even if minimal mode, WAL is required here if truncation happened after
+ * being created in the same transaction. It is not needed otherwise but
+ * we don't bother identifying the case precisely.
+ */
+ if (wstate->btws_use_wal ||
+ (blkno == BTREE_METAPAGE && BTPageGetMeta(page)->btm_root == 0))
{
/* We use the heap NEWPAGE record type for this */
log_newpage(&wstate->index->rd_node, MAIN_FORKNUM, blkno, page, true);
@@ -1055,6 +1061,11 @@ _bt_uppershutdown(BTWriteState *wstate, BTPageState *state)
* set to point to "P_NONE"). This changes the index to the "valid" state
* by filling in a valid magic number in the metapage.
*/
+ /*
+ * If no tuple was inserted, it's possible that we are truncating a
+ * relation. We need to emit WAL for the metapage in the case. However it
+ * is not required elsewise,
+ */
metapage = (Page) palloc(BLCKSZ);
_bt_initmetapage(metapage, rootblkno, rootlevel);
_bt_blwritepage(wstate, metapage, BTREE_METAPAGE);
--
2.16.3
----Next_Part(Thu_Oct_11_13_42_35_2018_969)----
^ permalink raw reply [nested|flat] 31+ messages in thread
* [PATCH 2/4] Write WAL for empty nbtree index build
@ 2018-10-11 01:03 Kyotaro Horiguchi <[email protected]>
0 siblings, 0 replies; 31+ messages in thread
From: Kyotaro Horiguchi @ 2018-10-11 01:03 UTC (permalink / raw)
After relation truncation indexes are also rebuild. It doesn't emit
WAL in minimal mode and if truncation happened within its creation
transaction, crash recovery leaves an empty index heap, which is
considered broken. This patch forces to emit WAL when an index_build
turns into empty nbtree index.
---
src/backend/access/nbtree/nbtsort.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index 16f5755777..2c2647b530 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -610,8 +610,14 @@ _bt_blwritepage(BTWriteState *wstate, Page page, BlockNumber blkno)
/* Ensure rd_smgr is open (could have been closed by relcache flush!) */
RelationOpenSmgr(wstate->index);
- /* XLOG stuff */
- if (wstate->btws_use_wal)
+ /* XLOG stuff
+ *
+ * Even if minimal mode, WAL is required here if truncation happened after
+ * being created in the same transaction. It is not needed otherwise but
+ * we don't bother identifying the case precisely.
+ */
+ if (wstate->btws_use_wal ||
+ (blkno == BTREE_METAPAGE && BTPageGetMeta(page)->btm_root == 0))
{
/* We use the heap NEWPAGE record type for this */
log_newpage(&wstate->index->rd_node, MAIN_FORKNUM, blkno, page, true);
@@ -1055,6 +1061,11 @@ _bt_uppershutdown(BTWriteState *wstate, BTPageState *state)
* set to point to "P_NONE"). This changes the index to the "valid" state
* by filling in a valid magic number in the metapage.
*/
+ /*
+ * If no tuple was inserted, it's possible that we are truncating a
+ * relation. We need to emit WAL for the metapage in the case. However it
+ * is not required elsewise,
+ */
metapage = (Page) palloc(BLCKSZ);
_bt_initmetapage(metapage, rootblkno, rootlevel);
_bt_blwritepage(wstate, metapage, BTREE_METAPAGE);
--
2.16.3
----Next_Part(Thu_Oct_11_17_04_53_2018_748)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="v3-0001-TAP-test-for-copy-truncation-optimization.patch"
^ permalink raw reply [nested|flat] 31+ messages in thread
* [PATCH 5/6] Preserve index dependencies on collation during pg_upgrade
@ 2019-11-07 06:33 Julien Rouhaud <[email protected]>
0 siblings, 0 replies; 31+ messages in thread
From: Julien Rouhaud @ 2019-11-07 06:33 UTC (permalink / raw)
A new binary_upgrade_set_index_coll_version() SQL function is added to override
the recorded dependency version for all or a single collation, for a specific
index.
Also teach pg_dump to call this function for all indexes, including indexes
created for constraints, when run with --binary-upgrade flag.
When pg_upgrade is run against an older version, collation versions are not
known and pg_dump will by default emit an alter index command to mark all
collation versions as unkown. However, it's possible that pg_upgrade is run
without upgrading the underlying collation libraries, so a new option
--collation-binary-compatible is added to avoid this behavior. This will
result in running pg_dump with a new --unknown-collations-binary-compatible
option, that can only be used in binary upgrade mode, to prevent pg_dump from
emitting call to this new function if the dependent collation version is
unknown. Note that if the collation version is known, this flag won't change
the behavior and the previous collation version will be preserved.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
src/backend/catalog/index.c | 62 ++++++
src/backend/utils/adt/pg_upgrade_support.c | 25 +++
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 192 +++++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++++++-----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 7 +
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/test/perl/PostgresNode.pm | 6 +-
15 files changed, 512 insertions(+), 72 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6629d736b8..b62a1d4eef 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ When upgrading from a PostgreSQL major version 12 or older, or from a
+ PostgreSQL major version that didn't have support for collation
+ versioning to a major version that now supports it, all indexes will be
+ marked as depending on an unknown collation version, as such versions
+ weren't tracked. As a result, numerous warning messages will be
+ emitted as it can be a sign of a corrupted index. If you're not
+ upgrading the collation libraries, and if you're absolutly certain that
+ all existing indexes are compatible with the current collation
+ libraries, you can use this flag to change this behavior and mark all
+ indexes as depending on current collation libraries version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9f9281a7af..2729a9d203 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3155,6 +3155,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 0d9e55cdcf..ea7c25ee35 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 81d6f3819a..81cd24c391 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -288,6 +289,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -388,6 +391,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -708,6 +712,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6857,7 +6865,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6893,7 +6903,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6918,7 +6983,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -6957,7 +7024,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -6992,7 +7061,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7023,7 +7094,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7057,7 +7130,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7097,6 +7172,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7122,6 +7199,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16390,10 +16469,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16453,6 +16533,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
}
}
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16482,6 +16566,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18439,6 +18538,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 21004e5078..6d1df24080 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -364,6 +364,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 4ef2036ecd..be2d137376 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 8750bfc36f..664bf42956 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index c619d02465..69d163f3cc 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8fb62c623..d201eb762b 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10200,6 +10200,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 9575268bd7..bba07a2319 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
--
2.20.1
--mP3DRpeJDSE+ciuQ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename*0=0006-Add-a-new-ALTER-INDEX-name-ALTER-COLLATION-name-REFR-v12;
filename*1=".patch"
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
@ 2023-06-13 10:26 John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: John Naylor @ 2023-06-13 10:26 UTC (permalink / raw)
To: YANG Xudong <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]
On Thu, Jun 8, 2023 at 12:24 PM YANG Xudong <[email protected]> wrote:
>
> This patch tries to add loongarch native crc32 check with crcc.*
> instructions to postgresql.
>
> The patch is tested on my Loongson 3A5000 machine with Loong Arch Linux
> and GCC 13.1.0 / clang 16.0.0 with
>
> - default ./configure
> - default meson setup
I took a quick look at this, and it seems mostly in line with other
architectures we support for CRC. I have a couple questions and comments:
configure.ac:
+AC_SUBST(CFLAGS_CRC)
This seems to be an unnecessary copy-paste. I think we only need one, after
all checks have run.
meson.build
+ if cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w,
__builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and
__builtin_loongarch_crcc_w_d_w without -march=loongarch64',
+ args: test_c_args)
+ # Use LoongArch CRC instruction unconditionally
+ cdata.set('USE_LOONGARCH_CRC32C', 1)
+ have_optimized_crc = true
+ elif cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w,
__builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and
__builtin_loongarch_crcc_w_d_w with -march=loongarch64',
+ args: test_c_args + ['-march=loongarch64'])
+ # Use LoongArch CRC instruction unconditionally
For x86 and Arm, if it fails to link without an -march flag, we allow for a
runtime check. The flags "-march=armv8-a+crc" and "-msse4.2" are for
instructions not found on all platforms. The patch also checks both ways,
and each one results in "Use LoongArch CRC instruction unconditionally".
The -march flag here is general, not specific. In other words, if this only
runs inside "+elif host_cpu == 'loongarch64'", why do we need both with
-march and without?
Also, I don't have a Loongarch machine for testing. Could you show that the
instructions are found in the binary, maybe using objdump and grep? Or a
performance test?
In the future, you may also consider running the buildfarm client on a
machine dedicated for testing. That will give us quick feedback if some
future new code doesn't work on this platform. More information here:
https://wiki.postgresql.org/wiki/PostgreSQL_Buildfarm_Howto
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
@ 2023-06-14 02:20 ` YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: YANG Xudong @ 2023-06-14 02:20 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]
Attached a new patch with fixes based on the comment below.
On 2023/6/13 18:26, John Naylor wrote:
>
> On Thu, Jun 8, 2023 at 12:24 PM YANG Xudong <[email protected]
> <mailto:[email protected]>> wrote:
> >
> > This patch tries to add loongarch native crc32 check with crcc.*
> > instructions to postgresql.
> >
> > The patch is tested on my Loongson 3A5000 machine with Loong Arch Linux
> > and GCC 13.1.0 / clang 16.0.0 with
> >
> > - default ./configure
> > - default meson setup
>
> I took a quick look at this, and it seems mostly in line with other
> architectures we support for CRC. I have a couple questions and comments:
>
> configure.ac <http://configure.ac;:
>
> +AC_SUBST(CFLAGS_CRC)
> > This seems to be an unnecessary copy-paste. I think we only need one,
> after all checks have run.
>
Removed the extra line.
> meson.build
>
> + if cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w,
> __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and
> __builtin_loongarch_crcc_w_d_w without -march=loongarch64',
> + args: test_c_args)
> + # Use LoongArch CRC instruction unconditionally
> + cdata.set('USE_LOONGARCH_CRC32C', 1)
> + have_optimized_crc = true
> + elif cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w,
> __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and
> __builtin_loongarch_crcc_w_d_w with -march=loongarch64',
> + args: test_c_args + ['-march=loongarch64'])
> + # Use LoongArch CRC instruction unconditionally
>
> For x86 and Arm, if it fails to link without an -march flag, we allow
> for a runtime check. The flags "-march=armv8-a+crc" and "-msse4.2" are
> for instructions not found on all platforms. The patch also checks both
> ways, and each one results in "Use LoongArch CRC instruction
> unconditionally". The -march flag here is general, not specific. In
> other words, if this only runs inside "+elif host_cpu == 'loongarch64'",
> why do we need both with -march and without?
>
Removed the elif branch.
> Also, I don't have a Loongarch machine for testing. Could you show that
> the instructions are found in the binary, maybe using objdump and grep?
> Or a performance test?
>
The output of the objdump command `objdump -dS
../postgres-build/tmp_install/usr/local/pgsql/bin/postgres | grep -B 30
-A 10 crcc` is attached.
Also the output of make check is attached.
I run a simple test program to compare the performance of
pg_comp_crc32c_loongarch and pg_comp_crc32c_sb8 on my test machine. The
result is that pg_comp_crc32c_loongarch is over 2x faster than
pg_comp_crc32c_sb8.
> In the future, you may also consider running the buildfarm client on a
> machine dedicated for testing. That will give us quick feedback if some
> future new code doesn't work on this platform. More information here:
>
> https://wiki.postgresql.org/wiki/PostgreSQL_Buildfarm_Howto
> <https://wiki.postgresql.org/wiki/PostgreSQL_Buildfarm_Howto;
>
I will contact the loongson community
(https://github.com/loongson-community) to see if they are able to
provide some machine for buildfarm or not.
> --
> John Naylor
> EDB: http://www.enterprisedb.com <http://www.enterprisedb.com;
--
YANG Xudong
From c6441c9e446d13ddf60b1f5432114a5289968403 Mon Sep 17 00:00:00 2001
From: YANG Xudong <[email protected]>
Date: Wed, 14 Jun 2023 09:49:49 +0800
Subject: [PATCH] Add loongarch native checksum implementation.
---
config/c-compiler.m4 | 34 ++++++++++
configure | 116 +++++++++++++++++++++++++++++++--
configure.ac | 37 ++++++++---
meson.build | 24 +++++++
src/include/pg_config.h.in | 3 +
src/include/port/pg_crc32c.h | 9 +++
src/port/Makefile | 5 ++
src/port/meson.build | 3 +
src/port/pg_crc32c_loongarch.c | 72 ++++++++++++++++++++
9 files changed, 288 insertions(+), 15 deletions(-)
create mode 100644 src/port/pg_crc32c_loongarch.c
diff --git a/config/c-compiler.m4 b/config/c-compiler.m4
index 5be8f0f08d..eb3af009c4 100644
--- a/config/c-compiler.m4
+++ b/config/c-compiler.m4
@@ -661,3 +661,37 @@ if test x"$Ac_cachevar" = x"yes"; then
fi
undefine([Ac_cachevar])dnl
])# PGAC_ARMV8_CRC32C_INTRINSICS
+
+# PGAC_LOONGARCH_CRC32C_INTRINSICS
+# ---------------------------
+# Check if the compiler supports the LoongArch CRCC instructions, using
+# __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w,
+# __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w
+# intrinsic functions.
+#
+# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics,
+# and CFLAGS_CRC.
+AC_DEFUN([PGAC_LOONGARCH_CRC32C_INTRINSICS],
+[define([Ac_cachevar], [AS_TR_SH([pgac_cv_loongarch_crc32c_intrinsics_$1])])dnl
+AC_CACHE_CHECK(
+ [for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=$1],
+ [Ac_cachevar],
+[pgac_save_CFLAGS=$CFLAGS
+CFLAGS="$pgac_save_CFLAGS $1"
+AC_LINK_IFELSE([AC_LANG_PROGRAM([],
+ [unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;])],
+ [Ac_cachevar=yes],
+ [Ac_cachevar=no])
+CFLAGS="$pgac_save_CFLAGS"])
+if test x"$Ac_cachevar" = x"yes"; then
+ CFLAGS_CRC="$1"
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+undefine([Ac_cachevar])dnl
+])# PGAC_LOONGARCH_CRC32C_INTRINSICS
diff --git a/configure b/configure
index 1b415142d1..fa4a0fdcb3 100755
--- a/configure
+++ b/configure
@@ -18114,6 +18114,94 @@ fi
fi
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+# CFLAGS_CRC is set if the extra flag is required.
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=" >&5
+$as_echo_n "checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=... " >&6; }
+if ${pgac_cv_loongarch_crc32c_intrinsics_+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ pgac_save_CFLAGS=$CFLAGS
+CFLAGS="$pgac_save_CFLAGS "
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+int
+main ()
+{
+unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ pgac_cv_loongarch_crc32c_intrinsics_=yes
+else
+ pgac_cv_loongarch_crc32c_intrinsics_=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+CFLAGS="$pgac_save_CFLAGS"
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_loongarch_crc32c_intrinsics_" >&5
+$as_echo "$pgac_cv_loongarch_crc32c_intrinsics_" >&6; }
+if test x"$pgac_cv_loongarch_crc32c_intrinsics_" = x"yes"; then
+ CFLAGS_CRC=""
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+
+if test x"$pgac_loongarch_crc32c_intrinsics" != x"yes"; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=-march=loongarch64" >&5
+$as_echo_n "checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=-march=loongarch64... " >&6; }
+if ${pgac_cv_loongarch_crc32c_intrinsics__march_loongarch64+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ pgac_save_CFLAGS=$CFLAGS
+CFLAGS="$pgac_save_CFLAGS -march=loongarch64"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+int
+main ()
+{
+unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ pgac_cv_loongarch_crc32c_intrinsics__march_loongarch64=yes
+else
+ pgac_cv_loongarch_crc32c_intrinsics__march_loongarch64=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+CFLAGS="$pgac_save_CFLAGS"
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_loongarch_crc32c_intrinsics__march_loongarch64" >&5
+$as_echo "$pgac_cv_loongarch_crc32c_intrinsics__march_loongarch64" >&6; }
+if test x"$pgac_cv_loongarch_crc32c_intrinsics__march_loongarch64" = x"yes"; then
+ CFLAGS_CRC="-march=loongarch64"
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+
+fi
+
# Select CRC-32C implementation.
@@ -18132,7 +18220,7 @@ fi
#
# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
-if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x""; then
+if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
USE_SSE42_CRC32C=1
@@ -18150,10 +18238,15 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
- # fall back to slicing-by-8 algorithm, which doesn't require any
- # special CPU support.
- USE_SLICING_BY_8_CRC32C=1
- fi
+ if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
+ # LoongArch CRCC instructions.
+ USE_LOONGARCH_CRC32C=1
+ else
+ # fall back to slicing-by-8 algorithm, which doesn't require any
+ # special CPU support.
+ USE_SLICING_BY_8_CRC32C=1
+ fi
+ fi
fi
fi
fi
@@ -18194,12 +18287,21 @@ $as_echo "#define USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK 1" >>confdefs.h
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: ARMv8 CRC instructions with runtime check" >&5
$as_echo "ARMv8 CRC instructions with runtime check" >&6; }
else
+ if test x"$USE_LOONGARCH_CRC32C" = x"1"; then
+
+$as_echo "#define USE_LOONGARCH_CRC32C 1" >>confdefs.h
+
+ PG_CRC32C_OBJS="pg_crc32c_loongarch.o"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: LoongArch CRCC instructions" >&5
+$as_echo "LoongArch CRCC instructions" >&6; }
+ else
$as_echo "#define USE_SLICING_BY_8_CRC32C 1" >>confdefs.h
- PG_CRC32C_OBJS="pg_crc32c_sb8.o"
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: slicing-by-8" >&5
+ PG_CRC32C_OBJS="pg_crc32c_sb8.o"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: slicing-by-8" >&5
$as_echo "slicing-by-8" >&6; }
+ fi
fi
fi
fi
diff --git a/configure.ac b/configure.ac
index 09558ada0f..a312a33eaf 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2126,6 +2126,16 @@ if test x"$pgac_armv8_crc32c_intrinsics" != x"yes"; then
PGAC_ARMV8_CRC32C_INTRINSICS([-march=armv8-a+crc])
fi
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+# CFLAGS_CRC is set if the extra flag is required.
+PGAC_LOONGARCH_CRC32C_INTRINSICS([])
+if test x"$pgac_loongarch_crc32c_intrinsics" != x"yes"; then
+ PGAC_LOONGARCH_CRC32C_INTRINSICS([-march=loongarch64])
+fi
+
AC_SUBST(CFLAGS_CRC)
# Select CRC-32C implementation.
@@ -2144,7 +2154,7 @@ AC_SUBST(CFLAGS_CRC)
#
# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
-if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x""; then
+if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
USE_SSE42_CRC32C=1
@@ -2162,10 +2172,15 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
- # fall back to slicing-by-8 algorithm, which doesn't require any
- # special CPU support.
- USE_SLICING_BY_8_CRC32C=1
- fi
+ if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
+ # LoongArch CRCC instructions.
+ USE_LOONGARCH_CRC32C=1
+ else
+ # fall back to slicing-by-8 algorithm, which doesn't require any
+ # special CPU support.
+ USE_SLICING_BY_8_CRC32C=1
+ fi
+ fi
fi
fi
fi
@@ -2193,9 +2208,15 @@ else
PG_CRC32C_OBJS="pg_crc32c_armv8.o pg_crc32c_sb8.o pg_crc32c_armv8_choose.o"
AC_MSG_RESULT(ARMv8 CRC instructions with runtime check)
else
- AC_DEFINE(USE_SLICING_BY_8_CRC32C, 1, [Define to 1 to use software CRC-32C implementation (slicing-by-8).])
- PG_CRC32C_OBJS="pg_crc32c_sb8.o"
- AC_MSG_RESULT(slicing-by-8)
+ if test x"$USE_LOONGARCH_CRC32C" = x"1"; then
+ AC_DEFINE(USE_LOONGARCH_CRC32C, 1, [Define to 1 to use LoongArch CRCC instructions.])
+ PG_CRC32C_OBJS="pg_crc32c_loongarch.o"
+ AC_MSG_RESULT(LoongArch CRCC instructions)
+ else
+ AC_DEFINE(USE_SLICING_BY_8_CRC32C, 1, [Define to 1 to use software CRC-32C implementation (slicing-by-8).])
+ PG_CRC32C_OBJS="pg_crc32c_sb8.o"
+ AC_MSG_RESULT(slicing-by-8)
+ fi
fi
fi
fi
diff --git a/meson.build b/meson.build
index 82f2782673..77a9b985df 100644
--- a/meson.build
+++ b/meson.build
@@ -2073,6 +2073,30 @@ int main(void)
cdata.set('USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK', 1)
have_optimized_crc = true
endif
+
+elif host_cpu == 'loongarch64'
+
+ prog = '''
+int main(void)
+{
+ unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+}
+'''
+
+ if cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and __builtin_loongarch_crcc_w_d_w',
+ args: test_c_args)
+ # Use LoongArch CRC instruction unconditionally
+ cdata.set('USE_LOONGARCH_CRC32C', 1)
+ have_optimized_crc = true
+ endif
+
endif
if not have_optimized_crc
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 6d572c3820..1f253566e4 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -724,6 +724,9 @@
/* Define to 1 to build with LLVM based JIT support. (--with-llvm) */
#undef USE_LLVM
+/* Define to 1 to use LoongArch CRCC instructions. */
+#undef USE_LOONGARCH_CRC32C
+
/* Define to 1 to build with LZ4 support. (--with-lz4) */
#undef USE_LZ4
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 7f8779261c..d085f1dc00 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -58,6 +58,15 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+#elif defined(USE_LOONGARCH_CRC32C)
+/* Use LoongArch CRCC instructions. */
+
+#define COMP_CRC32C(crc, data, len) \
+ ((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
+#define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
+
+extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+
#elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
/*
diff --git a/src/port/Makefile b/src/port/Makefile
index 711f59e32b..1933b0e750 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -98,6 +98,11 @@ pg_crc32c_armv8.o: CFLAGS+=$(CFLAGS_CRC)
pg_crc32c_armv8_shlib.o: CFLAGS+=$(CFLAGS_CRC)
pg_crc32c_armv8_srv.o: CFLAGS+=$(CFLAGS_CRC)
+# all versions of pg_crc32c_loongarch.o need CFLAGS_CRC
+pg_crc32c_loongarch.o: CFLAGS+=$(CFLAGS_CRC)
+pg_crc32c_loongarch_shlib.o: CFLAGS+=$(CFLAGS_CRC)
+pg_crc32c_loongarch_srv.o: CFLAGS+=$(CFLAGS_CRC)
+
#
# Shared library versions of object files
#
diff --git a/src/port/meson.build b/src/port/meson.build
index 24416b9bfc..3e77b2493a 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -92,6 +92,9 @@ replace_funcs_pos = [
['pg_crc32c_armv8_choose', 'USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK'],
['pg_crc32c_sb8', 'USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK'],
+ # loongarch
+ ['pg_crc32c_loongarch', 'USE_LOONGARCH_CRC32C'],
+
# generic fallback
['pg_crc32c_sb8', 'USE_SLICING_BY_8_CRC32C'],
]
diff --git a/src/port/pg_crc32c_loongarch.c b/src/port/pg_crc32c_loongarch.c
new file mode 100644
index 0000000000..b54fe12ff5
--- /dev/null
+++ b/src/port/pg_crc32c_loongarch.c
@@ -0,0 +1,72 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_crc32c_loongarch.c
+ * Compute CRC-32C checksum using LoongArch CRCC instructions
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_crc32c_loongarch.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "c.h"
+
+#include "port/pg_crc32c.h"
+
+pg_crc32c
+pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len)
+{
+ const unsigned char *p = data;
+ const unsigned char *pend = p + len;
+
+ /*
+ * Aligned memory access is significantly faster.
+ * Process leading bytes so that the loop below starts with a pointer aligned to eight bytes.
+ */
+ if (!PointerIsAligned(p, uint16) &&
+ p + 1 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
+ p += 1;
+ }
+ if (!PointerIsAligned(p, uint32) &&
+ p + 2 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
+ p += 2;
+ }
+ if (!PointerIsAligned(p, uint64) &&
+ p + 4 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
+ p += 4;
+ }
+
+ /* Process eight bytes at a time, as far as we can. */
+ while (p + 8 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_d_w(*(uint64 *) p, crc);
+ p += 8;
+ }
+
+ /* Process remaining 0-7 bytes. */
+ if (p + 4 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
+ p += 4;
+ }
+ if (p + 2 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
+ p += 2;
+ }
+ if (p < pend)
+ {
+ crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
+ }
+
+ return crc;
+}
--
2.41.0
62a548: 47fff9bf bnez $t1, -8(0x7ffff8) # 62a540 <strlcpy+0x2c>
;
}
return (s - src - 1); /* count does not include NUL */
62a54c: 00119584 sub.d $a0, $t0, $a1
}
62a550: 02fffc84 addi.d $a0, $a0, -1(0xfff)
62a554: 4c000020 jirl $zero, $ra, 0
000000000062a558 <pg_comp_crc32c_loongarch>:
/*
* Aligned memory access is significantly faster.
* Process leading bytes so that the loop below starts with a pointer aligned to eight bytes.
*/
if (!PointerIsAligned(p, uint16) &&
62a558: 034004ae andi $t2, $a1, 0x1
{
62a55c: 0015008d move $t1, $a0
const unsigned char *pend = p + len;
62a560: 001098a6 add.d $a2, $a1, $a2
if (!PointerIsAligned(p, uint16) &&
62a564: 001500ac move $t0, $a1
62a568: 40000dc0 beqz $t2, 12(0xc) # 62a574 <pg_comp_crc32c_loongarch+0x1c>
p + 1 <= pend)
62a56c: 02c004ae addi.d $t2, $a1, 1(0x1)
if (!PointerIsAligned(p, uint16) &&
62a570: 6c00ecce bgeu $a2, $t2, 236(0xec) # 62a65c <pg_comp_crc32c_loongarch+0x104>
{
crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
p += 1;
}
if (!PointerIsAligned(p, uint32) &&
62a574: 03400d8e andi $t2, $t0, 0x3
62a578: 40000dc0 beqz $t2, 12(0xc) # 62a584 <pg_comp_crc32c_loongarch+0x2c>
p + 2 <= pend)
62a57c: 02c008ae addi.d $t2, $a1, 2(0x2)
if (!PointerIsAligned(p, uint32) &&
62a580: 6c00c4ce bgeu $a2, $t2, 196(0xc4) # 62a644 <pg_comp_crc32c_loongarch+0xec>
{
crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
p += 2;
}
if (!PointerIsAligned(p, uint64) &&
62a584: 03401d8c andi $t0, $t0, 0x7
62a588: 40001d80 beqz $t0, 28(0x1c) # 62a5a4 <pg_comp_crc32c_loongarch+0x4c>
p + 4 <= pend)
62a58c: 02c010ac addi.d $t0, $a1, 4(0x4)
if (!PointerIsAligned(p, uint64) &&
62a590: 6800a4cc bltu $a2, $t0, 164(0xa4) # 62a634 <pg_comp_crc32c_loongarch+0xdc>
{
crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
62a594: 240000ae ldptr.w $t2, $a1, 0
p += 4;
62a598: 00150185 move $a1, $t0
crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
62a59c: 002735cc crcc.w.w.w $t0, $t2, $t1
62a5a0: 0040818d slli.w $t1, $t0, 0x0
}
/* Process eight bytes at a time, as far as we can. */
while (p + 8 <= pend)
62a5a4: 02c020ae addi.d $t2, $a1, 8(0x8)
62a5a8: 6800ccce bltu $a2, $t2, 204(0xcc) # 62a674 <pg_comp_crc32c_loongarch+0x11c>
p += 4;
62a5ac: 001501cc move $t0, $t2
{
crc = __builtin_loongarch_crcc_w_d_w(*(uint64 *) p, crc);
62a5b0: 260000a4 ldptr.d $a0, $a1, 0
62a5b4: 00150185 move $a1, $t0
while (p + 8 <= pend)
62a5b8: 02c0218c addi.d $t0, $t0, 8(0x8)
crc = __builtin_loongarch_crcc_w_d_w(*(uint64 *) p, crc);
62a5bc: 0027b484 crcc.w.d.w $a0, $a0, $t1
62a5c0: 0040808d slli.w $t1, $a0, 0x0
while (p + 8 <= pend)
62a5c4: 6fffeccc bgeu $a2, $t0, -20(0x3ffec) # 62a5b0 <pg_comp_crc32c_loongarch+0x58>
62a5c8: 0011b8c5 sub.d $a1, $a2, $t2
62a5cc: 00450ca5 srli.d $a1, $a1, 0x3
62a5d0: 02c004cc addi.d $t0, $a2, 1(0x1)
62a5d4: 02c005cf addi.d $t3, $t2, 1(0x1)
62a5d8: 00410ca5 slli.d $a1, $a1, 0x3
62a5dc: 0012bd8c sltu $t0, $t0, $t3
62a5e0: 0013b0a5 masknez $a1, $a1, $t0
62a5e4: 001095c5 add.d $a1, $t2, $a1
62a5e8: 001501a4 move $a0, $t1
p += 8;
}
/* Process remaining 0-7 bytes. */
if (p + 4 <= pend)
62a5ec: 02c010ac addi.d $t0, $a1, 4(0x4)
62a5f0: 680014cc bltu $a2, $t0, 20(0x14) # 62a604 <pg_comp_crc32c_loongarch+0xac>
{
crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
62a5f4: 240000a4 ldptr.w $a0, $a1, 0
p += 4;
62a5f8: 00150185 move $a1, $t0
crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
62a5fc: 00273484 crcc.w.w.w $a0, $a0, $t1
62a600: 00408084 slli.w $a0, $a0, 0x0
}
if (p + 2 <= pend)
62a604: 02c008ac addi.d $t0, $a1, 2(0x2)
62a608: 680014cc bltu $a2, $t0, 20(0x14) # 62a61c <pg_comp_crc32c_loongarch+0xc4>
{
crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
62a60c: 2a4000ad ld.hu $t1, $a1, 0
p += 2;
62a610: 00150185 move $a1, $t0
crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
62a614: 002691a4 crcc.w.h.w $a0, $t1, $a0
62a618: 00408084 slli.w $a0, $a0, 0x0
}
if (p < pend)
62a61c: 6c0014a6 bgeu $a1, $a2, 20(0x14) # 62a630 <pg_comp_crc32c_loongarch+0xd8>
{
crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
62a620: 2a0000ac ld.bu $t0, $a1, 0
62a624: 00261184 crcc.w.b.w $a0, $t0, $a0
62a628: 00408084 slli.w $a0, $a0, 0x0
}
return crc;
62a62c: 4c000020 jirl $zero, $ra, 0
}
62a630: 4c000020 jirl $zero, $ra, 0
while (p + 8 <= pend)
62a634: 02c020ae addi.d $t2, $a1, 8(0x8)
62a638: 001501a4 move $a0, $t1
62a63c: 6fff70ce bgeu $a2, $t2, -144(0x3ff70) # 62a5ac <pg_comp_crc32c_loongarch+0x54>
62a640: 53ffc7ff b -60(0xfffffc4) # 62a604 <pg_comp_crc32c_loongarch+0xac>
crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
62a644: 2a4000af ld.hu $t3, $a1, 0
p += 2;
62a648: 001501cc move $t0, $t2
if (!PointerIsAligned(p, uint64) &&
62a64c: 001501c5 move $a1, $t2
crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
62a650: 0026b5ed crcc.w.h.w $t1, $t3, $t1
62a654: 004081ad slli.w $t1, $t1, 0x0
p += 2;
62a658: 53ff2fff b -212(0xfffff2c) # 62a584 <pg_comp_crc32c_loongarch+0x2c>
crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
62a65c: 2a0000af ld.bu $t3, $a1, 0
p += 1;
62a660: 001501cc move $t0, $t2
if (!PointerIsAligned(p, uint32) &&
62a664: 001501c5 move $a1, $t2
crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
62a668: 002611ed crcc.w.b.w $t1, $t3, $a0
62a66c: 004081ad slli.w $t1, $t1, 0x0
p += 1;
62a670: 53ff07ff b -252(0xfffff04) # 62a574 <pg_comp_crc32c_loongarch+0x1c>
while (p + 8 <= pend)
62a674: 001501a4 move $a0, $t1
62a678: 53ff77ff b -140(0xfffff74) # 62a5ec <pg_comp_crc32c_loongarch+0x94>
000000000062a67c <bsearch_arg>:
void *
bsearch_arg(const void *key, const void *base0,
make -C ./src/backend generated-headers
make[1]: Entering directory '/home/yxd/playspace/postgres-build/src/backend'
make -C catalog distprep generated-header-symlinks
make -C nodes distprep generated-header-symlinks
make -C utils distprep generated-header-symlinks
prereqdir=`cd 'storage/lmgr/' >/dev/null && pwd` && \
cd '../../src/include/storage/' && rm -f lwlocknames.h && \
ln -s "$prereqdir/lwlocknames.h" .
make[2]: Entering directory '/home/yxd/playspace/postgres-build/src/backend/nodes'
make[2]: Nothing to be done for 'distprep'.
prereqdir=`cd './' >/dev/null && pwd` && \
cd '../../../src/include/nodes/' && for file in nodetags.h; do \
rm -f $file && ln -s "$prereqdir/$file" . ; \
done
make[2]: Entering directory '/home/yxd/playspace/postgres-build/src/backend/utils'
make[2]: Nothing to be done for 'distprep'.
prereqdir=`cd './' >/dev/null && pwd` && \
cd '../../../src/include/utils/' && for file in fmgroids.h fmgrprotos.h errcodes.h; do \
rm -f $file && ln -s "$prereqdir/$file" . ; \
done
sed -f /home/yxd/playspace/postgres-build/../postgres/src/backend/utils/Gen_dummy_probes.sed /home/yxd/playspace/postgres-build/../postgres/src/backend/utils/probes.d >probes.h
touch ../../../src/include/utils/header-stamp
touch ../../../src/include/nodes/header-stamp
make[2]: Leaving directory '/home/yxd/playspace/postgres-build/src/backend/nodes'
make[2]: Entering directory '/home/yxd/playspace/postgres-build/src/backend/catalog'
make[2]: Nothing to be done for 'distprep'.
prereqdir=`cd './' >/dev/null && pwd` && \
cd '../../../src/include/catalog/' && for file in pg_proc_d.h pg_type_d.h pg_attribute_d.h pg_class_d.h pg_attrdef_d.h pg_constraint_d.h pg_inherits_d.h pg_index_d.h pg_operator_d.h pg_opfamily_d.h pg_opclass_d.h pg_am_d.h pg_amop_d.h pg_amproc_d.h pg_language_d.h pg_largeobject_metadata_d.h pg_largeobject_d.h pg_aggregate_d.h pg_statistic_d.h pg_statistic_ext_d.h pg_statistic_ext_data_d.h pg_rewrite_d.h pg_trigger_d.h pg_event_trigger_d.h pg_description_d.h pg_cast_d.h pg_enum_d.h pg_namespace_d.h pg_conversion_d.h pg_depend_d.h pg_database_d.h pg_db_role_setting_d.h pg_tablespace_d.h pg_authid_d.h pg_auth_members_d.h pg_shdepend_d.h pg_shdescription_d.h pg_ts_config_d.h pg_ts_config_map_d.h pg_ts_dict_d.h pg_ts_parser_d.h pg_ts_template_d.h pg_extension_d.h pg_foreign_data_wrapper_d.h pg_foreign_server_d.h pg_user_mapping_d.h pg_foreign_table_d.h pg_policy_d.h pg_replication_origin_d.h pg_default_acl_d.h pg_init_privs_d.h pg_seclabel_d.h pg_shseclabel_d.h pg_collation_d.h pg_parameter_acl_d.h pg_partitioned_table_d.h pg_range_d.h pg_transform_d.h pg_sequence_d.h pg_publication_d.h pg_publication_namespace_d.h pg_publication_rel_d.h pg_subscription_d.h pg_subscription_rel_d.h schemapg.h system_fk_info.h; do \
rm -f $file && ln -s "$prereqdir/$file" . ; \
done
cd '../../../src/include/utils/' && rm -f probes.h && \
ln -s "../../../src/backend/utils/probes.h" .
make[2]: Leaving directory '/home/yxd/playspace/postgres-build/src/backend/utils'
touch ../../../src/include/catalog/header-stamp
make[2]: Leaving directory '/home/yxd/playspace/postgres-build/src/backend/catalog'
make[1]: Leaving directory '/home/yxd/playspace/postgres-build/src/backend'
rm -rf '/home/yxd/playspace/postgres-build'/tmp_install
/usr/bin/mkdir -p '/home/yxd/playspace/postgres-build'/tmp_install/log
make -C '.' DESTDIR='/home/yxd/playspace/postgres-build'/tmp_install install >'/home/yxd/playspace/postgres-build'/tmp_install/log/install.log 2>&1
make -j1 -C src/test/regress checkprep >>'/home/yxd/playspace/postgres-build'/tmp_install/log/install.log 2>&1
make -C src/test/regress check
make[1]: Entering directory '/home/yxd/playspace/postgres-build/src/test/regress'
make -C ../../../src/port all
make[2]: Entering directory '/home/yxd/playspace/postgres-build/src/port'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/yxd/playspace/postgres-build/src/port'
make -C ../../../src/common all
make[2]: Entering directory '/home/yxd/playspace/postgres-build/src/common'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/yxd/playspace/postgres-build/src/common'
make -C ../../../contrib/spi
make[2]: Entering directory '/home/yxd/playspace/postgres-build/contrib/spi'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/yxd/playspace/postgres-build/contrib/spi'
echo "# +++ regress check in src/test/regress +++" && PATH="/home/yxd/playspace/postgres-build/tmp_install/usr/local/pgsql/bin:/home/yxd/playspace/postgres-build/src/test/regress:$PATH" LD_LIBRARY_PATH="/home/yxd/playspace/postgres-build/tmp_install/usr/local/pgsql/lib" ../../../src/test/regress/pg_regress --temp-instance=./tmp_check --inputdir=/home/yxd/playspace/postgres-build/../postgres/src/test/regress --bindir= --dlpath=. --max-concurrent-tests=20 --schedule=/home/yxd/playspace/postgres-build/../postgres/src/test/regress/parallel_schedule
# +++ regress check in src/test/regress +++
# using temp instance on port 61696 with PID 815947
ok 1 - test_setup 284 ms
# parallel group (20 tests): int4 text txid char pg_lsn oid int2 name uuid regproc boolean varchar float8 float4 money int8 enum bit rangetypes numeric
ok 2 + boolean 196 ms
ok 3 + char 170 ms
ok 4 + name 179 ms
ok 5 + varchar 196 ms
ok 6 + text 149 ms
ok 7 + int2 178 ms
ok 8 + int4 130 ms
ok 9 + int8 210 ms
ok 10 + oid 177 ms
ok 11 + float4 201 ms
ok 12 + float8 196 ms
ok 13 + bit 229 ms
ok 14 + numeric 561 ms
ok 15 + txid 158 ms
ok 16 + uuid 186 ms
ok 17 + enum 221 ms
ok 18 + money 200 ms
ok 19 + rangetypes 531 ms
ok 20 + pg_lsn 175 ms
ok 21 + regproc 190 ms
# parallel group (20 tests): md5 circle line path macaddr8 numerology lseg point interval macaddr time timetz date inet strings box polygon multirangetypes timestamp timestamptz
ok 22 + strings 261 ms
ok 23 + md5 115 ms
ok 24 + numerology 157 ms
ok 25 + point 172 ms
ok 26 + lseg 167 ms
ok 27 + line 121 ms
ok 28 + box 361 ms
ok 29 + path 134 ms
ok 30 + polygon 383 ms
ok 31 + circle 119 ms
ok 32 + date 202 ms
ok 33 + time 179 ms
ok 34 + timetz 188 ms
ok 35 + timestamp 471 ms
ok 36 + timestamptz 482 ms
ok 37 + interval 172 ms
ok 38 + inet 229 ms
ok 39 + macaddr 177 ms
ok 40 + macaddr8 143 ms
ok 41 + multirangetypes 382 ms
# parallel group (12 tests): unicode misc_sanity expressions comments xid tstypes type_sanity horology geometry mvcc opr_sanity regex
ok 42 + geometry 195 ms
ok 43 + horology 193 ms
ok 44 + tstypes 159 ms
ok 45 + regex 471 ms
ok 46 + type_sanity 178 ms
ok 47 + opr_sanity 401 ms
ok 48 + misc_sanity 68 ms
ok 49 + comments 107 ms
ok 50 + expressions 104 ms
ok 51 + unicode 65 ms
ok 52 + xid 150 ms
ok 53 + mvcc 200 ms
# parallel group (5 tests): copydml copyselect copy insert_conflict insert
ok 54 + copy 80 ms
ok 55 + copyselect 59 ms
ok 56 + copydml 44 ms
ok 57 + insert 298 ms
ok 58 + insert_conflict 161 ms
# parallel group (7 tests): create_function_c create_type create_operator create_schema create_misc create_procedure create_table
ok 59 + create_function_c 48 ms
ok 60 + create_misc 82 ms
ok 61 + create_operator 60 ms
ok 62 + create_procedure 89 ms
ok 63 + create_table 393 ms
ok 64 + create_type 53 ms
ok 65 + create_schema 78 ms
# parallel group (5 tests): create_view index_including index_including_gist create_index_spgist create_index
ok 66 + create_index 737 ms
ok 67 + create_index_spgist 508 ms
ok 68 + create_view 299 ms
ok 69 + index_including 305 ms
ok 70 + index_including_gist 323 ms
# parallel group (16 tests): errors create_cast roleattributes create_aggregate hash_func drop_if_exists select create_function_sql typed_table create_am constraints infinite_recurse vacuum updatable_views inherit triggers
ok 71 + create_aggregate 162 ms
ok 72 + create_function_sql 284 ms
ok 73 + create_cast 131 ms
ok 74 + constraints 392 ms
ok 75 + triggers 1041 ms
ok 76 + select 234 ms
ok 77 + inherit 762 ms
ok 78 + typed_table 294 ms
ok 79 + vacuum 640 ms
ok 80 + drop_if_exists 215 ms
ok 81 + updatable_views 757 ms
ok 82 + roleattributes 142 ms
ok 83 + create_am 339 ms
ok 84 + hash_func 179 ms
ok 85 + errors 101 ms
ok 86 + infinite_recurse 503 ms
ok 87 - sanity_check 115 ms
# parallel group (20 tests): select_distinct_on namespace select_implicit delete case select_having random select_into prepared_xacts portals union transactions subselect select_distinct arrays update hash_index join aggregates btree_index
ok 88 + select_into 266 ms
ok 89 + select_distinct 517 ms
ok 90 + select_distinct_on 142 ms
ok 91 + select_implicit 197 ms
ok 92 + select_having 210 ms
ok 93 + subselect 500 ms
ok 94 + union 465 ms
ok 95 + case 207 ms
ok 96 + join 936 ms
ok 97 + aggregates 1133 ms
ok 98 + transactions 474 ms
ok 99 + random 233 ms
ok 100 + portals 380 ms
ok 101 + arrays 552 ms
ok 102 + btree_index 2003 ms
ok 103 + hash_index 932 ms
ok 104 + update 699 ms
ok 105 + delete 201 ms
ok 106 + namespace 159 ms
ok 107 + prepared_xacts 355 ms
# parallel group (20 tests): drop_operator init_privs security_label lock tablesample object_address collate password replica_identity identity groupingsets matview generated rowsecurity spgist gin gist brin join_hash privileges
ok 108 + brin 2754 ms
ok 109 + gin 1403 ms
ok 110 + gist 1441 ms
ok 111 + spgist 1249 ms
ok 112 + privileges 3351 ms
ok 113 + init_privs 137 ms
ok 114 + security_label 216 ms
ok 115 + collate 392 ms
ok 116 + matview 836 ms
ok 117 + lock 254 ms
ok 118 + replica_identity 492 ms
ok 119 + rowsecurity 1105 ms
ok 120 + object_address 339 ms
ok 121 + tablesample 298 ms
ok 122 + groupingsets 826 ms
ok 123 + drop_operator 118 ms
ok 124 + password 458 ms
ok 125 + identity 693 ms
ok 126 + generated 938 ms
ok 127 + join_hash 2758 ms
# parallel group (2 tests): brin_bloom brin_multi
ok 128 + brin_bloom 110 ms
ok 129 + brin_multi 246 ms
# parallel group (16 tests): dbsize tid async alter_operator tidrangescan tsrf tidscan sysviews create_role misc_functions alter_generic incremental_sort misc merge collate.icu.utf8 create_table_like
ok 130 + create_table_like 445 ms
ok 131 + alter_generic 265 ms
ok 132 + alter_operator 150 ms
ok 133 + misc 340 ms
ok 134 + async 135 ms
ok 135 + dbsize 109 ms
ok 136 + merge 369 ms
ok 137 + misc_functions 239 ms
ok 138 + sysviews 219 ms
ok 139 + tsrf 165 ms
ok 140 + tid 112 ms
ok 141 + tidscan 216 ms
ok 142 + tidrangescan 156 ms
ok 143 + collate.icu.utf8 395 ms
ok 144 + incremental_sort 323 ms
ok 145 + create_role 219 ms
# parallel group (7 tests): collate.linux.utf8 collate.windows.win1252 amutils psql_crosstab rules psql stats_ext
ok 146 + rules 335 ms
ok 147 + psql 356 ms
ok 148 + psql_crosstab 67 ms
ok 149 + amutils 58 ms
ok 150 + stats_ext 1821 ms
ok 151 + collate.linux.utf8 30 ms
ok 152 + collate.windows.win1252 30 ms
ok 153 - select_parallel 1919 ms
ok 154 - write_parallel 240 ms
ok 155 - vacuum_parallel 159 ms
# parallel group (2 tests): subscription publication
ok 156 + publication 338 ms
ok 157 + subscription 65 ms
# parallel group (17 tests): xmlmap combocid advisory_lock equivclass portals_p2 guc functional_deps tsdicts dependency select_views window bitmapops cluster tsearch foreign_data indirect_toast foreign_key
ok 158 + select_views 352 ms
ok 159 + portals_p2 194 ms
ok 160 + foreign_key 1250 ms
ok 161 + cluster 626 ms
ok 162 + dependency 260 ms
ok 163 + guc 210 ms
ok 164 + bitmapops 582 ms
ok 165 + combocid 164 ms
ok 166 + tsearch 645 ms
ok 167 + tsdicts 248 ms
ok 168 + foreign_data 830 ms
ok 169 + window 525 ms
ok 170 + xmlmap 162 ms
ok 171 + functional_deps 241 ms
ok 172 + advisory_lock 185 ms
ok 173 + indirect_toast 893 ms
ok 174 + equivclass 186 ms
# parallel group (7 tests): json_encoding jsonpath jsonpath_encoding sqljson jsonb_jsonpath json jsonb
ok 175 + json 153 ms
ok 176 + jsonb 266 ms
ok 177 + json_encoding 56 ms
ok 178 + jsonpath 58 ms
ok 179 + jsonpath_encoding 65 ms
ok 180 + jsonb_jsonpath 131 ms
ok 181 + sqljson 116 ms
# parallel group (18 tests): prepare returning plancache xml limit conversion copy2 polymorphism rowtypes sequence with temp rangefuncs truncate largeobject domain plpgsql alter_table
ok 182 + plancache 310 ms
ok 183 + limit 329 ms
ok 184 + plpgsql 889 ms
ok 185 + copy2 456 ms
ok 186 + temp 541 ms
ok 187 + domain 599 ms
ok 188 + rangefuncs 552 ms
ok 189 + prepare 179 ms
ok 190 + conversion 402 ms
ok 191 + truncate 552 ms
ok 192 + alter_table 1563 ms
ok 193 + sequence 507 ms
ok 194 + polymorphism 482 ms
ok 195 + rowtypes 490 ms
ok 196 + returning 245 ms
ok 197 + largeobject 592 ms
ok 198 + with 518 ms
ok 199 + xml 313 ms
# parallel group (12 tests): hash_part reloptions partition_info explain compression memoize indexing partition_join stats partition_aggregate partition_prune tuplesort
ok 200 + partition_join 1295 ms
ok 201 + partition_prune 1589 ms
ok 202 + reloptions 186 ms
ok 203 + hash_part 166 ms
ok 204 + indexing 1225 ms
ok 205 + partition_aggregate 1382 ms
ok 206 + partition_info 187 ms
ok 207 + tuplesort 2142 ms
ok 208 + explain 243 ms
ok 209 + compression 307 ms
ok 210 + memoize 317 ms
ok 211 + stats 1321 ms
# parallel group (2 tests): event_trigger oidjoins
ok 212 + event_trigger 126 ms
ok 213 + oidjoins 240 ms
ok 214 - fast_default 144 ms
ok 215 - tablespace 308 ms
1..215
# All 215 tests passed.
make[1]: Leaving directory '/home/yxd/playspace/postgres-build/src/test/regress'
Attachments:
[text/plain] 0001-Add-loongarch-native-checksum-implementation.patch (16.6K, ../../[email protected]/2-0001-Add-loongarch-native-checksum-implementation.patch)
download | inline diff:
From c6441c9e446d13ddf60b1f5432114a5289968403 Mon Sep 17 00:00:00 2001
From: YANG Xudong <[email protected]>
Date: Wed, 14 Jun 2023 09:49:49 +0800
Subject: [PATCH] Add loongarch native checksum implementation.
---
config/c-compiler.m4 | 34 ++++++++++
configure | 116 +++++++++++++++++++++++++++++++--
configure.ac | 37 ++++++++---
meson.build | 24 +++++++
src/include/pg_config.h.in | 3 +
src/include/port/pg_crc32c.h | 9 +++
src/port/Makefile | 5 ++
src/port/meson.build | 3 +
src/port/pg_crc32c_loongarch.c | 72 ++++++++++++++++++++
9 files changed, 288 insertions(+), 15 deletions(-)
create mode 100644 src/port/pg_crc32c_loongarch.c
diff --git a/config/c-compiler.m4 b/config/c-compiler.m4
index 5be8f0f08d..eb3af009c4 100644
--- a/config/c-compiler.m4
+++ b/config/c-compiler.m4
@@ -661,3 +661,37 @@ if test x"$Ac_cachevar" = x"yes"; then
fi
undefine([Ac_cachevar])dnl
])# PGAC_ARMV8_CRC32C_INTRINSICS
+
+# PGAC_LOONGARCH_CRC32C_INTRINSICS
+# ---------------------------
+# Check if the compiler supports the LoongArch CRCC instructions, using
+# __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w,
+# __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w
+# intrinsic functions.
+#
+# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics,
+# and CFLAGS_CRC.
+AC_DEFUN([PGAC_LOONGARCH_CRC32C_INTRINSICS],
+[define([Ac_cachevar], [AS_TR_SH([pgac_cv_loongarch_crc32c_intrinsics_$1])])dnl
+AC_CACHE_CHECK(
+ [for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=$1],
+ [Ac_cachevar],
+[pgac_save_CFLAGS=$CFLAGS
+CFLAGS="$pgac_save_CFLAGS $1"
+AC_LINK_IFELSE([AC_LANG_PROGRAM([],
+ [unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;])],
+ [Ac_cachevar=yes],
+ [Ac_cachevar=no])
+CFLAGS="$pgac_save_CFLAGS"])
+if test x"$Ac_cachevar" = x"yes"; then
+ CFLAGS_CRC="$1"
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+undefine([Ac_cachevar])dnl
+])# PGAC_LOONGARCH_CRC32C_INTRINSICS
diff --git a/configure b/configure
index 1b415142d1..fa4a0fdcb3 100755
--- a/configure
+++ b/configure
@@ -18114,6 +18114,94 @@ fi
fi
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+# CFLAGS_CRC is set if the extra flag is required.
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=" >&5
+$as_echo_n "checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=... " >&6; }
+if ${pgac_cv_loongarch_crc32c_intrinsics_+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ pgac_save_CFLAGS=$CFLAGS
+CFLAGS="$pgac_save_CFLAGS "
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+int
+main ()
+{
+unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ pgac_cv_loongarch_crc32c_intrinsics_=yes
+else
+ pgac_cv_loongarch_crc32c_intrinsics_=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+CFLAGS="$pgac_save_CFLAGS"
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_loongarch_crc32c_intrinsics_" >&5
+$as_echo "$pgac_cv_loongarch_crc32c_intrinsics_" >&6; }
+if test x"$pgac_cv_loongarch_crc32c_intrinsics_" = x"yes"; then
+ CFLAGS_CRC=""
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+
+if test x"$pgac_loongarch_crc32c_intrinsics" != x"yes"; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=-march=loongarch64" >&5
+$as_echo_n "checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=-march=loongarch64... " >&6; }
+if ${pgac_cv_loongarch_crc32c_intrinsics__march_loongarch64+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ pgac_save_CFLAGS=$CFLAGS
+CFLAGS="$pgac_save_CFLAGS -march=loongarch64"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+int
+main ()
+{
+unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ pgac_cv_loongarch_crc32c_intrinsics__march_loongarch64=yes
+else
+ pgac_cv_loongarch_crc32c_intrinsics__march_loongarch64=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+CFLAGS="$pgac_save_CFLAGS"
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_loongarch_crc32c_intrinsics__march_loongarch64" >&5
+$as_echo "$pgac_cv_loongarch_crc32c_intrinsics__march_loongarch64" >&6; }
+if test x"$pgac_cv_loongarch_crc32c_intrinsics__march_loongarch64" = x"yes"; then
+ CFLAGS_CRC="-march=loongarch64"
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+
+fi
+
# Select CRC-32C implementation.
@@ -18132,7 +18220,7 @@ fi
#
# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
-if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x""; then
+if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
USE_SSE42_CRC32C=1
@@ -18150,10 +18238,15 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
- # fall back to slicing-by-8 algorithm, which doesn't require any
- # special CPU support.
- USE_SLICING_BY_8_CRC32C=1
- fi
+ if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
+ # LoongArch CRCC instructions.
+ USE_LOONGARCH_CRC32C=1
+ else
+ # fall back to slicing-by-8 algorithm, which doesn't require any
+ # special CPU support.
+ USE_SLICING_BY_8_CRC32C=1
+ fi
+ fi
fi
fi
fi
@@ -18194,12 +18287,21 @@ $as_echo "#define USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK 1" >>confdefs.h
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: ARMv8 CRC instructions with runtime check" >&5
$as_echo "ARMv8 CRC instructions with runtime check" >&6; }
else
+ if test x"$USE_LOONGARCH_CRC32C" = x"1"; then
+
+$as_echo "#define USE_LOONGARCH_CRC32C 1" >>confdefs.h
+
+ PG_CRC32C_OBJS="pg_crc32c_loongarch.o"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: LoongArch CRCC instructions" >&5
+$as_echo "LoongArch CRCC instructions" >&6; }
+ else
$as_echo "#define USE_SLICING_BY_8_CRC32C 1" >>confdefs.h
- PG_CRC32C_OBJS="pg_crc32c_sb8.o"
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: slicing-by-8" >&5
+ PG_CRC32C_OBJS="pg_crc32c_sb8.o"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: slicing-by-8" >&5
$as_echo "slicing-by-8" >&6; }
+ fi
fi
fi
fi
diff --git a/configure.ac b/configure.ac
index 09558ada0f..a312a33eaf 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2126,6 +2126,16 @@ if test x"$pgac_armv8_crc32c_intrinsics" != x"yes"; then
PGAC_ARMV8_CRC32C_INTRINSICS([-march=armv8-a+crc])
fi
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+# CFLAGS_CRC is set if the extra flag is required.
+PGAC_LOONGARCH_CRC32C_INTRINSICS([])
+if test x"$pgac_loongarch_crc32c_intrinsics" != x"yes"; then
+ PGAC_LOONGARCH_CRC32C_INTRINSICS([-march=loongarch64])
+fi
+
AC_SUBST(CFLAGS_CRC)
# Select CRC-32C implementation.
@@ -2144,7 +2154,7 @@ AC_SUBST(CFLAGS_CRC)
#
# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
-if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x""; then
+if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
USE_SSE42_CRC32C=1
@@ -2162,10 +2172,15 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
- # fall back to slicing-by-8 algorithm, which doesn't require any
- # special CPU support.
- USE_SLICING_BY_8_CRC32C=1
- fi
+ if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
+ # LoongArch CRCC instructions.
+ USE_LOONGARCH_CRC32C=1
+ else
+ # fall back to slicing-by-8 algorithm, which doesn't require any
+ # special CPU support.
+ USE_SLICING_BY_8_CRC32C=1
+ fi
+ fi
fi
fi
fi
@@ -2193,9 +2208,15 @@ else
PG_CRC32C_OBJS="pg_crc32c_armv8.o pg_crc32c_sb8.o pg_crc32c_armv8_choose.o"
AC_MSG_RESULT(ARMv8 CRC instructions with runtime check)
else
- AC_DEFINE(USE_SLICING_BY_8_CRC32C, 1, [Define to 1 to use software CRC-32C implementation (slicing-by-8).])
- PG_CRC32C_OBJS="pg_crc32c_sb8.o"
- AC_MSG_RESULT(slicing-by-8)
+ if test x"$USE_LOONGARCH_CRC32C" = x"1"; then
+ AC_DEFINE(USE_LOONGARCH_CRC32C, 1, [Define to 1 to use LoongArch CRCC instructions.])
+ PG_CRC32C_OBJS="pg_crc32c_loongarch.o"
+ AC_MSG_RESULT(LoongArch CRCC instructions)
+ else
+ AC_DEFINE(USE_SLICING_BY_8_CRC32C, 1, [Define to 1 to use software CRC-32C implementation (slicing-by-8).])
+ PG_CRC32C_OBJS="pg_crc32c_sb8.o"
+ AC_MSG_RESULT(slicing-by-8)
+ fi
fi
fi
fi
diff --git a/meson.build b/meson.build
index 82f2782673..77a9b985df 100644
--- a/meson.build
+++ b/meson.build
@@ -2073,6 +2073,30 @@ int main(void)
cdata.set('USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK', 1)
have_optimized_crc = true
endif
+
+elif host_cpu == 'loongarch64'
+
+ prog = '''
+int main(void)
+{
+ unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+}
+'''
+
+ if cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and __builtin_loongarch_crcc_w_d_w',
+ args: test_c_args)
+ # Use LoongArch CRC instruction unconditionally
+ cdata.set('USE_LOONGARCH_CRC32C', 1)
+ have_optimized_crc = true
+ endif
+
endif
if not have_optimized_crc
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 6d572c3820..1f253566e4 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -724,6 +724,9 @@
/* Define to 1 to build with LLVM based JIT support. (--with-llvm) */
#undef USE_LLVM
+/* Define to 1 to use LoongArch CRCC instructions. */
+#undef USE_LOONGARCH_CRC32C
+
/* Define to 1 to build with LZ4 support. (--with-lz4) */
#undef USE_LZ4
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 7f8779261c..d085f1dc00 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -58,6 +58,15 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+#elif defined(USE_LOONGARCH_CRC32C)
+/* Use LoongArch CRCC instructions. */
+
+#define COMP_CRC32C(crc, data, len) \
+ ((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
+#define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
+
+extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+
#elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
/*
diff --git a/src/port/Makefile b/src/port/Makefile
index 711f59e32b..1933b0e750 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -98,6 +98,11 @@ pg_crc32c_armv8.o: CFLAGS+=$(CFLAGS_CRC)
pg_crc32c_armv8_shlib.o: CFLAGS+=$(CFLAGS_CRC)
pg_crc32c_armv8_srv.o: CFLAGS+=$(CFLAGS_CRC)
+# all versions of pg_crc32c_loongarch.o need CFLAGS_CRC
+pg_crc32c_loongarch.o: CFLAGS+=$(CFLAGS_CRC)
+pg_crc32c_loongarch_shlib.o: CFLAGS+=$(CFLAGS_CRC)
+pg_crc32c_loongarch_srv.o: CFLAGS+=$(CFLAGS_CRC)
+
#
# Shared library versions of object files
#
diff --git a/src/port/meson.build b/src/port/meson.build
index 24416b9bfc..3e77b2493a 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -92,6 +92,9 @@ replace_funcs_pos = [
['pg_crc32c_armv8_choose', 'USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK'],
['pg_crc32c_sb8', 'USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK'],
+ # loongarch
+ ['pg_crc32c_loongarch', 'USE_LOONGARCH_CRC32C'],
+
# generic fallback
['pg_crc32c_sb8', 'USE_SLICING_BY_8_CRC32C'],
]
diff --git a/src/port/pg_crc32c_loongarch.c b/src/port/pg_crc32c_loongarch.c
new file mode 100644
index 0000000000..b54fe12ff5
--- /dev/null
+++ b/src/port/pg_crc32c_loongarch.c
@@ -0,0 +1,72 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_crc32c_loongarch.c
+ * Compute CRC-32C checksum using LoongArch CRCC instructions
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_crc32c_loongarch.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "c.h"
+
+#include "port/pg_crc32c.h"
+
+pg_crc32c
+pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len)
+{
+ const unsigned char *p = data;
+ const unsigned char *pend = p + len;
+
+ /*
+ * Aligned memory access is significantly faster.
+ * Process leading bytes so that the loop below starts with a pointer aligned to eight bytes.
+ */
+ if (!PointerIsAligned(p, uint16) &&
+ p + 1 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
+ p += 1;
+ }
+ if (!PointerIsAligned(p, uint32) &&
+ p + 2 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
+ p += 2;
+ }
+ if (!PointerIsAligned(p, uint64) &&
+ p + 4 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
+ p += 4;
+ }
+
+ /* Process eight bytes at a time, as far as we can. */
+ while (p + 8 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_d_w(*(uint64 *) p, crc);
+ p += 8;
+ }
+
+ /* Process remaining 0-7 bytes. */
+ if (p + 4 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
+ p += 4;
+ }
+ if (p + 2 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
+ p += 2;
+ }
+ if (p < pend)
+ {
+ crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
+ }
+
+ return crc;
+}
--
2.41.0
[text/plain] objdump.out (6.2K, ../../[email protected]/3-objdump.out)
download | inline:
62a548: 47fff9bf bnez $t1, -8(0x7ffff8) # 62a540 <strlcpy+0x2c>
;
}
return (s - src - 1); /* count does not include NUL */
62a54c: 00119584 sub.d $a0, $t0, $a1
}
62a550: 02fffc84 addi.d $a0, $a0, -1(0xfff)
62a554: 4c000020 jirl $zero, $ra, 0
000000000062a558 <pg_comp_crc32c_loongarch>:
/*
* Aligned memory access is significantly faster.
* Process leading bytes so that the loop below starts with a pointer aligned to eight bytes.
*/
if (!PointerIsAligned(p, uint16) &&
62a558: 034004ae andi $t2, $a1, 0x1
{
62a55c: 0015008d move $t1, $a0
const unsigned char *pend = p + len;
62a560: 001098a6 add.d $a2, $a1, $a2
if (!PointerIsAligned(p, uint16) &&
62a564: 001500ac move $t0, $a1
62a568: 40000dc0 beqz $t2, 12(0xc) # 62a574 <pg_comp_crc32c_loongarch+0x1c>
p + 1 <= pend)
62a56c: 02c004ae addi.d $t2, $a1, 1(0x1)
if (!PointerIsAligned(p, uint16) &&
62a570: 6c00ecce bgeu $a2, $t2, 236(0xec) # 62a65c <pg_comp_crc32c_loongarch+0x104>
{
crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
p += 1;
}
if (!PointerIsAligned(p, uint32) &&
62a574: 03400d8e andi $t2, $t0, 0x3
62a578: 40000dc0 beqz $t2, 12(0xc) # 62a584 <pg_comp_crc32c_loongarch+0x2c>
p + 2 <= pend)
62a57c: 02c008ae addi.d $t2, $a1, 2(0x2)
if (!PointerIsAligned(p, uint32) &&
62a580: 6c00c4ce bgeu $a2, $t2, 196(0xc4) # 62a644 <pg_comp_crc32c_loongarch+0xec>
{
crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
p += 2;
}
if (!PointerIsAligned(p, uint64) &&
62a584: 03401d8c andi $t0, $t0, 0x7
62a588: 40001d80 beqz $t0, 28(0x1c) # 62a5a4 <pg_comp_crc32c_loongarch+0x4c>
p + 4 <= pend)
62a58c: 02c010ac addi.d $t0, $a1, 4(0x4)
if (!PointerIsAligned(p, uint64) &&
62a590: 6800a4cc bltu $a2, $t0, 164(0xa4) # 62a634 <pg_comp_crc32c_loongarch+0xdc>
{
crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
62a594: 240000ae ldptr.w $t2, $a1, 0
p += 4;
62a598: 00150185 move $a1, $t0
crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
62a59c: 002735cc crcc.w.w.w $t0, $t2, $t1
62a5a0: 0040818d slli.w $t1, $t0, 0x0
}
/* Process eight bytes at a time, as far as we can. */
while (p + 8 <= pend)
62a5a4: 02c020ae addi.d $t2, $a1, 8(0x8)
62a5a8: 6800ccce bltu $a2, $t2, 204(0xcc) # 62a674 <pg_comp_crc32c_loongarch+0x11c>
p += 4;
62a5ac: 001501cc move $t0, $t2
{
crc = __builtin_loongarch_crcc_w_d_w(*(uint64 *) p, crc);
62a5b0: 260000a4 ldptr.d $a0, $a1, 0
62a5b4: 00150185 move $a1, $t0
while (p + 8 <= pend)
62a5b8: 02c0218c addi.d $t0, $t0, 8(0x8)
crc = __builtin_loongarch_crcc_w_d_w(*(uint64 *) p, crc);
62a5bc: 0027b484 crcc.w.d.w $a0, $a0, $t1
62a5c0: 0040808d slli.w $t1, $a0, 0x0
while (p + 8 <= pend)
62a5c4: 6fffeccc bgeu $a2, $t0, -20(0x3ffec) # 62a5b0 <pg_comp_crc32c_loongarch+0x58>
62a5c8: 0011b8c5 sub.d $a1, $a2, $t2
62a5cc: 00450ca5 srli.d $a1, $a1, 0x3
62a5d0: 02c004cc addi.d $t0, $a2, 1(0x1)
62a5d4: 02c005cf addi.d $t3, $t2, 1(0x1)
62a5d8: 00410ca5 slli.d $a1, $a1, 0x3
62a5dc: 0012bd8c sltu $t0, $t0, $t3
62a5e0: 0013b0a5 masknez $a1, $a1, $t0
62a5e4: 001095c5 add.d $a1, $t2, $a1
62a5e8: 001501a4 move $a0, $t1
p += 8;
}
/* Process remaining 0-7 bytes. */
if (p + 4 <= pend)
62a5ec: 02c010ac addi.d $t0, $a1, 4(0x4)
62a5f0: 680014cc bltu $a2, $t0, 20(0x14) # 62a604 <pg_comp_crc32c_loongarch+0xac>
{
crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
62a5f4: 240000a4 ldptr.w $a0, $a1, 0
p += 4;
62a5f8: 00150185 move $a1, $t0
crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
62a5fc: 00273484 crcc.w.w.w $a0, $a0, $t1
62a600: 00408084 slli.w $a0, $a0, 0x0
}
if (p + 2 <= pend)
62a604: 02c008ac addi.d $t0, $a1, 2(0x2)
62a608: 680014cc bltu $a2, $t0, 20(0x14) # 62a61c <pg_comp_crc32c_loongarch+0xc4>
{
crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
62a60c: 2a4000ad ld.hu $t1, $a1, 0
p += 2;
62a610: 00150185 move $a1, $t0
crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
62a614: 002691a4 crcc.w.h.w $a0, $t1, $a0
62a618: 00408084 slli.w $a0, $a0, 0x0
}
if (p < pend)
62a61c: 6c0014a6 bgeu $a1, $a2, 20(0x14) # 62a630 <pg_comp_crc32c_loongarch+0xd8>
{
crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
62a620: 2a0000ac ld.bu $t0, $a1, 0
62a624: 00261184 crcc.w.b.w $a0, $t0, $a0
62a628: 00408084 slli.w $a0, $a0, 0x0
}
return crc;
62a62c: 4c000020 jirl $zero, $ra, 0
}
62a630: 4c000020 jirl $zero, $ra, 0
while (p + 8 <= pend)
62a634: 02c020ae addi.d $t2, $a1, 8(0x8)
62a638: 001501a4 move $a0, $t1
62a63c: 6fff70ce bgeu $a2, $t2, -144(0x3ff70) # 62a5ac <pg_comp_crc32c_loongarch+0x54>
62a640: 53ffc7ff b -60(0xfffffc4) # 62a604 <pg_comp_crc32c_loongarch+0xac>
crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
62a644: 2a4000af ld.hu $t3, $a1, 0
p += 2;
62a648: 001501cc move $t0, $t2
if (!PointerIsAligned(p, uint64) &&
62a64c: 001501c5 move $a1, $t2
crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
62a650: 0026b5ed crcc.w.h.w $t1, $t3, $t1
62a654: 004081ad slli.w $t1, $t1, 0x0
p += 2;
62a658: 53ff2fff b -212(0xfffff2c) # 62a584 <pg_comp_crc32c_loongarch+0x2c>
crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
62a65c: 2a0000af ld.bu $t3, $a1, 0
p += 1;
62a660: 001501cc move $t0, $t2
if (!PointerIsAligned(p, uint32) &&
62a664: 001501c5 move $a1, $t2
crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
62a668: 002611ed crcc.w.b.w $t1, $t3, $a0
62a66c: 004081ad slli.w $t1, $t1, 0x0
p += 1;
62a670: 53ff07ff b -252(0xfffff04) # 62a574 <pg_comp_crc32c_loongarch+0x1c>
while (p + 8 <= pend)
62a674: 001501a4 move $a0, $t1
62a678: 53ff77ff b -140(0xfffff74) # 62a5ec <pg_comp_crc32c_loongarch+0x94>
000000000062a67c <bsearch_arg>:
void *
bsearch_arg(const void *key, const void *base0,
[text/plain] test.out (21.0K, ../../[email protected]/4-test.out)
download | inline:
make -C ./src/backend generated-headers
make[1]: Entering directory '/home/yxd/playspace/postgres-build/src/backend'
make -C catalog distprep generated-header-symlinks
make -C nodes distprep generated-header-symlinks
make -C utils distprep generated-header-symlinks
prereqdir=`cd 'storage/lmgr/' >/dev/null && pwd` && \
cd '../../src/include/storage/' && rm -f lwlocknames.h && \
ln -s "$prereqdir/lwlocknames.h" .
make[2]: Entering directory '/home/yxd/playspace/postgres-build/src/backend/nodes'
make[2]: Nothing to be done for 'distprep'.
prereqdir=`cd './' >/dev/null && pwd` && \
cd '../../../src/include/nodes/' && for file in nodetags.h; do \
rm -f $file && ln -s "$prereqdir/$file" . ; \
done
make[2]: Entering directory '/home/yxd/playspace/postgres-build/src/backend/utils'
make[2]: Nothing to be done for 'distprep'.
prereqdir=`cd './' >/dev/null && pwd` && \
cd '../../../src/include/utils/' && for file in fmgroids.h fmgrprotos.h errcodes.h; do \
rm -f $file && ln -s "$prereqdir/$file" . ; \
done
sed -f /home/yxd/playspace/postgres-build/../postgres/src/backend/utils/Gen_dummy_probes.sed /home/yxd/playspace/postgres-build/../postgres/src/backend/utils/probes.d >probes.h
touch ../../../src/include/utils/header-stamp
touch ../../../src/include/nodes/header-stamp
make[2]: Leaving directory '/home/yxd/playspace/postgres-build/src/backend/nodes'
make[2]: Entering directory '/home/yxd/playspace/postgres-build/src/backend/catalog'
make[2]: Nothing to be done for 'distprep'.
prereqdir=`cd './' >/dev/null && pwd` && \
cd '../../../src/include/catalog/' && for file in pg_proc_d.h pg_type_d.h pg_attribute_d.h pg_class_d.h pg_attrdef_d.h pg_constraint_d.h pg_inherits_d.h pg_index_d.h pg_operator_d.h pg_opfamily_d.h pg_opclass_d.h pg_am_d.h pg_amop_d.h pg_amproc_d.h pg_language_d.h pg_largeobject_metadata_d.h pg_largeobject_d.h pg_aggregate_d.h pg_statistic_d.h pg_statistic_ext_d.h pg_statistic_ext_data_d.h pg_rewrite_d.h pg_trigger_d.h pg_event_trigger_d.h pg_description_d.h pg_cast_d.h pg_enum_d.h pg_namespace_d.h pg_conversion_d.h pg_depend_d.h pg_database_d.h pg_db_role_setting_d.h pg_tablespace_d.h pg_authid_d.h pg_auth_members_d.h pg_shdepend_d.h pg_shdescription_d.h pg_ts_config_d.h pg_ts_config_map_d.h pg_ts_dict_d.h pg_ts_parser_d.h pg_ts_template_d.h pg_extension_d.h pg_foreign_data_wrapper_d.h pg_foreign_server_d.h pg_user_mapping_d.h pg_foreign_table_d.h pg_policy_d.h pg_replication_origin_d.h pg_default_acl_d.h pg_init_privs_d.h pg_seclabel_d.h pg_shseclabel_d.h pg_collation_d.h pg_parameter_acl_d.h pg_partitioned_table_d.h pg_range_d.h pg_transform_d.h pg_sequence_d.h pg_publication_d.h pg_publication_namespace_d.h pg_publication_rel_d.h pg_subscription_d.h pg_subscription_rel_d.h schemapg.h system_fk_info.h; do \
rm -f $file && ln -s "$prereqdir/$file" . ; \
done
cd '../../../src/include/utils/' && rm -f probes.h && \
ln -s "../../../src/backend/utils/probes.h" .
make[2]: Leaving directory '/home/yxd/playspace/postgres-build/src/backend/utils'
touch ../../../src/include/catalog/header-stamp
make[2]: Leaving directory '/home/yxd/playspace/postgres-build/src/backend/catalog'
make[1]: Leaving directory '/home/yxd/playspace/postgres-build/src/backend'
rm -rf '/home/yxd/playspace/postgres-build'/tmp_install
/usr/bin/mkdir -p '/home/yxd/playspace/postgres-build'/tmp_install/log
make -C '.' DESTDIR='/home/yxd/playspace/postgres-build'/tmp_install install >'/home/yxd/playspace/postgres-build'/tmp_install/log/install.log 2>&1
make -j1 -C src/test/regress checkprep >>'/home/yxd/playspace/postgres-build'/tmp_install/log/install.log 2>&1
make -C src/test/regress check
make[1]: Entering directory '/home/yxd/playspace/postgres-build/src/test/regress'
make -C ../../../src/port all
make[2]: Entering directory '/home/yxd/playspace/postgres-build/src/port'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/yxd/playspace/postgres-build/src/port'
make -C ../../../src/common all
make[2]: Entering directory '/home/yxd/playspace/postgres-build/src/common'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/yxd/playspace/postgres-build/src/common'
make -C ../../../contrib/spi
make[2]: Entering directory '/home/yxd/playspace/postgres-build/contrib/spi'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/yxd/playspace/postgres-build/contrib/spi'
echo "# +++ regress check in src/test/regress +++" && PATH="/home/yxd/playspace/postgres-build/tmp_install/usr/local/pgsql/bin:/home/yxd/playspace/postgres-build/src/test/regress:$PATH" LD_LIBRARY_PATH="/home/yxd/playspace/postgres-build/tmp_install/usr/local/pgsql/lib" ../../../src/test/regress/pg_regress --temp-instance=./tmp_check --inputdir=/home/yxd/playspace/postgres-build/../postgres/src/test/regress --bindir= --dlpath=. --max-concurrent-tests=20 --schedule=/home/yxd/playspace/postgres-build/../postgres/src/test/regress/parallel_schedule
# +++ regress check in src/test/regress +++
# using temp instance on port 61696 with PID 815947
ok 1 - test_setup 284 ms
# parallel group (20 tests): int4 text txid char pg_lsn oid int2 name uuid regproc boolean varchar float8 float4 money int8 enum bit rangetypes numeric
ok 2 + boolean 196 ms
ok 3 + char 170 ms
ok 4 + name 179 ms
ok 5 + varchar 196 ms
ok 6 + text 149 ms
ok 7 + int2 178 ms
ok 8 + int4 130 ms
ok 9 + int8 210 ms
ok 10 + oid 177 ms
ok 11 + float4 201 ms
ok 12 + float8 196 ms
ok 13 + bit 229 ms
ok 14 + numeric 561 ms
ok 15 + txid 158 ms
ok 16 + uuid 186 ms
ok 17 + enum 221 ms
ok 18 + money 200 ms
ok 19 + rangetypes 531 ms
ok 20 + pg_lsn 175 ms
ok 21 + regproc 190 ms
# parallel group (20 tests): md5 circle line path macaddr8 numerology lseg point interval macaddr time timetz date inet strings box polygon multirangetypes timestamp timestamptz
ok 22 + strings 261 ms
ok 23 + md5 115 ms
ok 24 + numerology 157 ms
ok 25 + point 172 ms
ok 26 + lseg 167 ms
ok 27 + line 121 ms
ok 28 + box 361 ms
ok 29 + path 134 ms
ok 30 + polygon 383 ms
ok 31 + circle 119 ms
ok 32 + date 202 ms
ok 33 + time 179 ms
ok 34 + timetz 188 ms
ok 35 + timestamp 471 ms
ok 36 + timestamptz 482 ms
ok 37 + interval 172 ms
ok 38 + inet 229 ms
ok 39 + macaddr 177 ms
ok 40 + macaddr8 143 ms
ok 41 + multirangetypes 382 ms
# parallel group (12 tests): unicode misc_sanity expressions comments xid tstypes type_sanity horology geometry mvcc opr_sanity regex
ok 42 + geometry 195 ms
ok 43 + horology 193 ms
ok 44 + tstypes 159 ms
ok 45 + regex 471 ms
ok 46 + type_sanity 178 ms
ok 47 + opr_sanity 401 ms
ok 48 + misc_sanity 68 ms
ok 49 + comments 107 ms
ok 50 + expressions 104 ms
ok 51 + unicode 65 ms
ok 52 + xid 150 ms
ok 53 + mvcc 200 ms
# parallel group (5 tests): copydml copyselect copy insert_conflict insert
ok 54 + copy 80 ms
ok 55 + copyselect 59 ms
ok 56 + copydml 44 ms
ok 57 + insert 298 ms
ok 58 + insert_conflict 161 ms
# parallel group (7 tests): create_function_c create_type create_operator create_schema create_misc create_procedure create_table
ok 59 + create_function_c 48 ms
ok 60 + create_misc 82 ms
ok 61 + create_operator 60 ms
ok 62 + create_procedure 89 ms
ok 63 + create_table 393 ms
ok 64 + create_type 53 ms
ok 65 + create_schema 78 ms
# parallel group (5 tests): create_view index_including index_including_gist create_index_spgist create_index
ok 66 + create_index 737 ms
ok 67 + create_index_spgist 508 ms
ok 68 + create_view 299 ms
ok 69 + index_including 305 ms
ok 70 + index_including_gist 323 ms
# parallel group (16 tests): errors create_cast roleattributes create_aggregate hash_func drop_if_exists select create_function_sql typed_table create_am constraints infinite_recurse vacuum updatable_views inherit triggers
ok 71 + create_aggregate 162 ms
ok 72 + create_function_sql 284 ms
ok 73 + create_cast 131 ms
ok 74 + constraints 392 ms
ok 75 + triggers 1041 ms
ok 76 + select 234 ms
ok 77 + inherit 762 ms
ok 78 + typed_table 294 ms
ok 79 + vacuum 640 ms
ok 80 + drop_if_exists 215 ms
ok 81 + updatable_views 757 ms
ok 82 + roleattributes 142 ms
ok 83 + create_am 339 ms
ok 84 + hash_func 179 ms
ok 85 + errors 101 ms
ok 86 + infinite_recurse 503 ms
ok 87 - sanity_check 115 ms
# parallel group (20 tests): select_distinct_on namespace select_implicit delete case select_having random select_into prepared_xacts portals union transactions subselect select_distinct arrays update hash_index join aggregates btree_index
ok 88 + select_into 266 ms
ok 89 + select_distinct 517 ms
ok 90 + select_distinct_on 142 ms
ok 91 + select_implicit 197 ms
ok 92 + select_having 210 ms
ok 93 + subselect 500 ms
ok 94 + union 465 ms
ok 95 + case 207 ms
ok 96 + join 936 ms
ok 97 + aggregates 1133 ms
ok 98 + transactions 474 ms
ok 99 + random 233 ms
ok 100 + portals 380 ms
ok 101 + arrays 552 ms
ok 102 + btree_index 2003 ms
ok 103 + hash_index 932 ms
ok 104 + update 699 ms
ok 105 + delete 201 ms
ok 106 + namespace 159 ms
ok 107 + prepared_xacts 355 ms
# parallel group (20 tests): drop_operator init_privs security_label lock tablesample object_address collate password replica_identity identity groupingsets matview generated rowsecurity spgist gin gist brin join_hash privileges
ok 108 + brin 2754 ms
ok 109 + gin 1403 ms
ok 110 + gist 1441 ms
ok 111 + spgist 1249 ms
ok 112 + privileges 3351 ms
ok 113 + init_privs 137 ms
ok 114 + security_label 216 ms
ok 115 + collate 392 ms
ok 116 + matview 836 ms
ok 117 + lock 254 ms
ok 118 + replica_identity 492 ms
ok 119 + rowsecurity 1105 ms
ok 120 + object_address 339 ms
ok 121 + tablesample 298 ms
ok 122 + groupingsets 826 ms
ok 123 + drop_operator 118 ms
ok 124 + password 458 ms
ok 125 + identity 693 ms
ok 126 + generated 938 ms
ok 127 + join_hash 2758 ms
# parallel group (2 tests): brin_bloom brin_multi
ok 128 + brin_bloom 110 ms
ok 129 + brin_multi 246 ms
# parallel group (16 tests): dbsize tid async alter_operator tidrangescan tsrf tidscan sysviews create_role misc_functions alter_generic incremental_sort misc merge collate.icu.utf8 create_table_like
ok 130 + create_table_like 445 ms
ok 131 + alter_generic 265 ms
ok 132 + alter_operator 150 ms
ok 133 + misc 340 ms
ok 134 + async 135 ms
ok 135 + dbsize 109 ms
ok 136 + merge 369 ms
ok 137 + misc_functions 239 ms
ok 138 + sysviews 219 ms
ok 139 + tsrf 165 ms
ok 140 + tid 112 ms
ok 141 + tidscan 216 ms
ok 142 + tidrangescan 156 ms
ok 143 + collate.icu.utf8 395 ms
ok 144 + incremental_sort 323 ms
ok 145 + create_role 219 ms
# parallel group (7 tests): collate.linux.utf8 collate.windows.win1252 amutils psql_crosstab rules psql stats_ext
ok 146 + rules 335 ms
ok 147 + psql 356 ms
ok 148 + psql_crosstab 67 ms
ok 149 + amutils 58 ms
ok 150 + stats_ext 1821 ms
ok 151 + collate.linux.utf8 30 ms
ok 152 + collate.windows.win1252 30 ms
ok 153 - select_parallel 1919 ms
ok 154 - write_parallel 240 ms
ok 155 - vacuum_parallel 159 ms
# parallel group (2 tests): subscription publication
ok 156 + publication 338 ms
ok 157 + subscription 65 ms
# parallel group (17 tests): xmlmap combocid advisory_lock equivclass portals_p2 guc functional_deps tsdicts dependency select_views window bitmapops cluster tsearch foreign_data indirect_toast foreign_key
ok 158 + select_views 352 ms
ok 159 + portals_p2 194 ms
ok 160 + foreign_key 1250 ms
ok 161 + cluster 626 ms
ok 162 + dependency 260 ms
ok 163 + guc 210 ms
ok 164 + bitmapops 582 ms
ok 165 + combocid 164 ms
ok 166 + tsearch 645 ms
ok 167 + tsdicts 248 ms
ok 168 + foreign_data 830 ms
ok 169 + window 525 ms
ok 170 + xmlmap 162 ms
ok 171 + functional_deps 241 ms
ok 172 + advisory_lock 185 ms
ok 173 + indirect_toast 893 ms
ok 174 + equivclass 186 ms
# parallel group (7 tests): json_encoding jsonpath jsonpath_encoding sqljson jsonb_jsonpath json jsonb
ok 175 + json 153 ms
ok 176 + jsonb 266 ms
ok 177 + json_encoding 56 ms
ok 178 + jsonpath 58 ms
ok 179 + jsonpath_encoding 65 ms
ok 180 + jsonb_jsonpath 131 ms
ok 181 + sqljson 116 ms
# parallel group (18 tests): prepare returning plancache xml limit conversion copy2 polymorphism rowtypes sequence with temp rangefuncs truncate largeobject domain plpgsql alter_table
ok 182 + plancache 310 ms
ok 183 + limit 329 ms
ok 184 + plpgsql 889 ms
ok 185 + copy2 456 ms
ok 186 + temp 541 ms
ok 187 + domain 599 ms
ok 188 + rangefuncs 552 ms
ok 189 + prepare 179 ms
ok 190 + conversion 402 ms
ok 191 + truncate 552 ms
ok 192 + alter_table 1563 ms
ok 193 + sequence 507 ms
ok 194 + polymorphism 482 ms
ok 195 + rowtypes 490 ms
ok 196 + returning 245 ms
ok 197 + largeobject 592 ms
ok 198 + with 518 ms
ok 199 + xml 313 ms
# parallel group (12 tests): hash_part reloptions partition_info explain compression memoize indexing partition_join stats partition_aggregate partition_prune tuplesort
ok 200 + partition_join 1295 ms
ok 201 + partition_prune 1589 ms
ok 202 + reloptions 186 ms
ok 203 + hash_part 166 ms
ok 204 + indexing 1225 ms
ok 205 + partition_aggregate 1382 ms
ok 206 + partition_info 187 ms
ok 207 + tuplesort 2142 ms
ok 208 + explain 243 ms
ok 209 + compression 307 ms
ok 210 + memoize 317 ms
ok 211 + stats 1321 ms
# parallel group (2 tests): event_trigger oidjoins
ok 212 + event_trigger 126 ms
ok 213 + oidjoins 240 ms
ok 214 - fast_default 144 ms
ok 215 - tablespace 308 ms
1..215
# All 215 tests passed.
make[1]: Leaving directory '/home/yxd/playspace/postgres-build/src/test/regress'
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
@ 2023-06-15 10:30 ` John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: John Naylor @ 2023-06-15 10:30 UTC (permalink / raw)
To: YANG Xudong <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]
On Wed, Jun 14, 2023 at 9:20 AM YANG Xudong <[email protected]> wrote:
>
> Attached a new patch with fixes based on the comment below.
Note: It's helpful to pass "-v" to git format-patch, to have different
versions.
> > For x86 and Arm, if it fails to link without an -march flag, we allow
> > for a runtime check. The flags "-march=armv8-a+crc" and "-msse4.2" are
> > for instructions not found on all platforms. The patch also checks both
> > ways, and each one results in "Use LoongArch CRC instruction
> > unconditionally". The -march flag here is general, not specific. In
> > other words, if this only runs inside "+elif host_cpu == 'loongarch64'",
> > why do we need both with -march and without?
> >
>
> Removed the elif branch.
Okay, since we've confirmed that no arch flag is necessary, some other
places can be simplified:
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -98,6 +98,11 @@ pg_crc32c_armv8.o: CFLAGS+=$(CFLAGS_CRC)
pg_crc32c_armv8_shlib.o: CFLAGS+=$(CFLAGS_CRC)
pg_crc32c_armv8_srv.o: CFLAGS+=$(CFLAGS_CRC)
+# all versions of pg_crc32c_loongarch.o need CFLAGS_CRC
+pg_crc32c_loongarch.o: CFLAGS+=$(CFLAGS_CRC)
+pg_crc32c_loongarch_shlib.o: CFLAGS+=$(CFLAGS_CRC)
+pg_crc32c_loongarch_srv.o: CFLAGS+=$(CFLAGS_CRC)
This was copy-and-pasted from platforms that use a runtime check, so should
be unnecessary.
+# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics,
+# and CFLAGS_CRC.
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+# CFLAGS_CRC is set if the extra flag is required.
Same here -- it seems we don't need to set CFLAGS_CRC at all. Can you
confirm?
> > Also, I don't have a Loongarch machine for testing. Could you show that
> > the instructions are found in the binary, maybe using objdump and grep?
> > Or a performance test?
> >
>
> The output of the objdump command `objdump -dS
> ../postgres-build/tmp_install/usr/local/pgsql/bin/postgres | grep -B 30
> -A 10 crcc` is attached.
Thanks for confirming.
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
@ 2023-06-16 01:28 ` YANG Xudong <[email protected]>
2023-07-05 02:15 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-07 11:01 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
0 siblings, 2 replies; 31+ messages in thread
From: YANG Xudong @ 2023-06-16 01:28 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]
Updated the patch based on the comments.
On 2023/6/15 18:30, John Naylor wrote:
>
> On Wed, Jun 14, 2023 at 9:20 AM YANG Xudong <[email protected]
> <mailto:[email protected]>> wrote:
> >
> > Attached a new patch with fixes based on the comment below.
>
> Note: It's helpful to pass "-v" to git format-patch, to have different
> versions.
>
Added v2
> > > For x86 and Arm, if it fails to link without an -march flag, we allow
> > > for a runtime check. The flags "-march=armv8-a+crc" and "-msse4.2" are
> > > for instructions not found on all platforms. The patch also checks both
> > > ways, and each one results in "Use LoongArch CRC instruction
> > > unconditionally". The -march flag here is general, not specific. In
> > > other words, if this only runs inside "+elif host_cpu ==
> 'loongarch64'",
> > > why do we need both with -march and without?
> > >
> >
> > Removed the elif branch.
>
> Okay, since we've confirmed that no arch flag is necessary, some other
> places can be simplified:
>
> --- a/src/port/Makefile
> +++ b/src/port/Makefile
> @@ -98,6 +98,11 @@ pg_crc32c_armv8.o: CFLAGS+=$(CFLAGS_CRC)
> pg_crc32c_armv8_shlib.o: CFLAGS+=$(CFLAGS_CRC)
> pg_crc32c_armv8_srv.o: CFLAGS+=$(CFLAGS_CRC)
>
> +# all versions of pg_crc32c_loongarch.o need CFLAGS_CRC
> +pg_crc32c_loongarch.o: CFLAGS+=$(CFLAGS_CRC)
> +pg_crc32c_loongarch_shlib.o: CFLAGS+=$(CFLAGS_CRC)
> +pg_crc32c_loongarch_srv.o: CFLAGS+=$(CFLAGS_CRC)
>
> This was copy-and-pasted from platforms that use a runtime check, so
> should be unnecessary.
>
Removed these lines.
> +# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics,
> +# and CFLAGS_CRC.
>
> +# Check if __builtin_loongarch_crcc_* intrinsics can be used
> +# with the default compiler flags.
> +# CFLAGS_CRC is set if the extra flag is required.
>
> Same here -- it seems we don't need to set CFLAGS_CRC at all. Can you
> confirm?
>
We don't need to set CFLAGS_CRC as commented. I have updated the
configure script to make it align with the logic in meson build script.
> > > Also, I don't have a Loongarch machine for testing. Could you show that
> > > the instructions are found in the binary, maybe using objdump and grep?
> > > Or a performance test?
> > >
> >
> > The output of the objdump command `objdump -dS
> > ../postgres-build/tmp_install/usr/local/pgsql/bin/postgres | grep -B 30
> > -A 10 crcc` is attached.
>
> Thanks for confirming.
>
> --
> John Naylor
> EDB: http://www.enterprisedb.com <http://www.enterprisedb.com;
From b2329478b5331e2aa9942c7ee4e23e3bfa871c1d Mon Sep 17 00:00:00 2001
From: YANG Xudong <[email protected]>
Date: Fri, 16 Jun 2023 09:22:08 +0800
Subject: [PATCH v2] Add loongarch native checksum implementation.
---
config/c-compiler.m4 | 34 ++++++++++++++++
configure | 73 ++++++++++++++++++++++++++++++----
configure.ac | 33 +++++++++++----
meson.build | 24 +++++++++++
src/include/pg_config.h.in | 3 ++
src/include/port/pg_crc32c.h | 9 +++++
src/port/meson.build | 3 ++
src/port/pg_crc32c_loongarch.c | 72 +++++++++++++++++++++++++++++++++
8 files changed, 236 insertions(+), 15 deletions(-)
create mode 100644 src/port/pg_crc32c_loongarch.c
diff --git a/config/c-compiler.m4 b/config/c-compiler.m4
index 5be8f0f08d..eb3af009c4 100644
--- a/config/c-compiler.m4
+++ b/config/c-compiler.m4
@@ -661,3 +661,37 @@ if test x"$Ac_cachevar" = x"yes"; then
fi
undefine([Ac_cachevar])dnl
])# PGAC_ARMV8_CRC32C_INTRINSICS
+
+# PGAC_LOONGARCH_CRC32C_INTRINSICS
+# ---------------------------
+# Check if the compiler supports the LoongArch CRCC instructions, using
+# __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w,
+# __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w
+# intrinsic functions.
+#
+# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics,
+# and CFLAGS_CRC.
+AC_DEFUN([PGAC_LOONGARCH_CRC32C_INTRINSICS],
+[define([Ac_cachevar], [AS_TR_SH([pgac_cv_loongarch_crc32c_intrinsics_$1])])dnl
+AC_CACHE_CHECK(
+ [for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=$1],
+ [Ac_cachevar],
+[pgac_save_CFLAGS=$CFLAGS
+CFLAGS="$pgac_save_CFLAGS $1"
+AC_LINK_IFELSE([AC_LANG_PROGRAM([],
+ [unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;])],
+ [Ac_cachevar=yes],
+ [Ac_cachevar=no])
+CFLAGS="$pgac_save_CFLAGS"])
+if test x"$Ac_cachevar" = x"yes"; then
+ CFLAGS_CRC="$1"
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+undefine([Ac_cachevar])dnl
+])# PGAC_LOONGARCH_CRC32C_INTRINSICS
diff --git a/configure b/configure
index 1b415142d1..7c60a26c11 100755
--- a/configure
+++ b/configure
@@ -18114,6 +18114,51 @@ fi
fi
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=" >&5
+$as_echo_n "checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=... " >&6; }
+if ${pgac_cv_loongarch_crc32c_intrinsics_+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ pgac_save_CFLAGS=$CFLAGS
+CFLAGS="$pgac_save_CFLAGS "
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+int
+main ()
+{
+unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ pgac_cv_loongarch_crc32c_intrinsics_=yes
+else
+ pgac_cv_loongarch_crc32c_intrinsics_=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+CFLAGS="$pgac_save_CFLAGS"
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_loongarch_crc32c_intrinsics_" >&5
+$as_echo "$pgac_cv_loongarch_crc32c_intrinsics_" >&6; }
+if test x"$pgac_cv_loongarch_crc32c_intrinsics_" = x"yes"; then
+ CFLAGS_CRC=""
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+
+
# Select CRC-32C implementation.
@@ -18132,7 +18177,7 @@ fi
#
# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
-if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x""; then
+if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
USE_SSE42_CRC32C=1
@@ -18150,10 +18195,15 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
- # fall back to slicing-by-8 algorithm, which doesn't require any
- # special CPU support.
- USE_SLICING_BY_8_CRC32C=1
- fi
+ if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
+ # LoongArch CRCC instructions.
+ USE_LOONGARCH_CRC32C=1
+ else
+ # fall back to slicing-by-8 algorithm, which doesn't require any
+ # special CPU support.
+ USE_SLICING_BY_8_CRC32C=1
+ fi
+ fi
fi
fi
fi
@@ -18194,12 +18244,21 @@ $as_echo "#define USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK 1" >>confdefs.h
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: ARMv8 CRC instructions with runtime check" >&5
$as_echo "ARMv8 CRC instructions with runtime check" >&6; }
else
+ if test x"$USE_LOONGARCH_CRC32C" = x"1"; then
+
+$as_echo "#define USE_LOONGARCH_CRC32C 1" >>confdefs.h
+
+ PG_CRC32C_OBJS="pg_crc32c_loongarch.o"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: LoongArch CRCC instructions" >&5
+$as_echo "LoongArch CRCC instructions" >&6; }
+ else
$as_echo "#define USE_SLICING_BY_8_CRC32C 1" >>confdefs.h
- PG_CRC32C_OBJS="pg_crc32c_sb8.o"
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: slicing-by-8" >&5
+ PG_CRC32C_OBJS="pg_crc32c_sb8.o"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: slicing-by-8" >&5
$as_echo "slicing-by-8" >&6; }
+ fi
fi
fi
fi
diff --git a/configure.ac b/configure.ac
index 09558ada0f..d44e9f9ef8 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2126,6 +2126,12 @@ if test x"$pgac_armv8_crc32c_intrinsics" != x"yes"; then
PGAC_ARMV8_CRC32C_INTRINSICS([-march=armv8-a+crc])
fi
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+PGAC_LOONGARCH_CRC32C_INTRINSICS([])
+
AC_SUBST(CFLAGS_CRC)
# Select CRC-32C implementation.
@@ -2144,7 +2150,7 @@ AC_SUBST(CFLAGS_CRC)
#
# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
-if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x""; then
+if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
USE_SSE42_CRC32C=1
@@ -2162,10 +2168,15 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
- # fall back to slicing-by-8 algorithm, which doesn't require any
- # special CPU support.
- USE_SLICING_BY_8_CRC32C=1
- fi
+ if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
+ # LoongArch CRCC instructions.
+ USE_LOONGARCH_CRC32C=1
+ else
+ # fall back to slicing-by-8 algorithm, which doesn't require any
+ # special CPU support.
+ USE_SLICING_BY_8_CRC32C=1
+ fi
+ fi
fi
fi
fi
@@ -2193,9 +2204,15 @@ else
PG_CRC32C_OBJS="pg_crc32c_armv8.o pg_crc32c_sb8.o pg_crc32c_armv8_choose.o"
AC_MSG_RESULT(ARMv8 CRC instructions with runtime check)
else
- AC_DEFINE(USE_SLICING_BY_8_CRC32C, 1, [Define to 1 to use software CRC-32C implementation (slicing-by-8).])
- PG_CRC32C_OBJS="pg_crc32c_sb8.o"
- AC_MSG_RESULT(slicing-by-8)
+ if test x"$USE_LOONGARCH_CRC32C" = x"1"; then
+ AC_DEFINE(USE_LOONGARCH_CRC32C, 1, [Define to 1 to use LoongArch CRCC instructions.])
+ PG_CRC32C_OBJS="pg_crc32c_loongarch.o"
+ AC_MSG_RESULT(LoongArch CRCC instructions)
+ else
+ AC_DEFINE(USE_SLICING_BY_8_CRC32C, 1, [Define to 1 to use software CRC-32C implementation (slicing-by-8).])
+ PG_CRC32C_OBJS="pg_crc32c_sb8.o"
+ AC_MSG_RESULT(slicing-by-8)
+ fi
fi
fi
fi
diff --git a/meson.build b/meson.build
index 82f2782673..77a9b985df 100644
--- a/meson.build
+++ b/meson.build
@@ -2073,6 +2073,30 @@ int main(void)
cdata.set('USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK', 1)
have_optimized_crc = true
endif
+
+elif host_cpu == 'loongarch64'
+
+ prog = '''
+int main(void)
+{
+ unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+}
+'''
+
+ if cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and __builtin_loongarch_crcc_w_d_w',
+ args: test_c_args)
+ # Use LoongArch CRC instruction unconditionally
+ cdata.set('USE_LOONGARCH_CRC32C', 1)
+ have_optimized_crc = true
+ endif
+
endif
if not have_optimized_crc
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 6d572c3820..1f253566e4 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -724,6 +724,9 @@
/* Define to 1 to build with LLVM based JIT support. (--with-llvm) */
#undef USE_LLVM
+/* Define to 1 to use LoongArch CRCC instructions. */
+#undef USE_LOONGARCH_CRC32C
+
/* Define to 1 to build with LZ4 support. (--with-lz4) */
#undef USE_LZ4
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 7f8779261c..d085f1dc00 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -58,6 +58,15 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+#elif defined(USE_LOONGARCH_CRC32C)
+/* Use LoongArch CRCC instructions. */
+
+#define COMP_CRC32C(crc, data, len) \
+ ((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
+#define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
+
+extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+
#elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
/*
diff --git a/src/port/meson.build b/src/port/meson.build
index 24416b9bfc..3e77b2493a 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -92,6 +92,9 @@ replace_funcs_pos = [
['pg_crc32c_armv8_choose', 'USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK'],
['pg_crc32c_sb8', 'USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK'],
+ # loongarch
+ ['pg_crc32c_loongarch', 'USE_LOONGARCH_CRC32C'],
+
# generic fallback
['pg_crc32c_sb8', 'USE_SLICING_BY_8_CRC32C'],
]
diff --git a/src/port/pg_crc32c_loongarch.c b/src/port/pg_crc32c_loongarch.c
new file mode 100644
index 0000000000..b54fe12ff5
--- /dev/null
+++ b/src/port/pg_crc32c_loongarch.c
@@ -0,0 +1,72 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_crc32c_loongarch.c
+ * Compute CRC-32C checksum using LoongArch CRCC instructions
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_crc32c_loongarch.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "c.h"
+
+#include "port/pg_crc32c.h"
+
+pg_crc32c
+pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len)
+{
+ const unsigned char *p = data;
+ const unsigned char *pend = p + len;
+
+ /*
+ * Aligned memory access is significantly faster.
+ * Process leading bytes so that the loop below starts with a pointer aligned to eight bytes.
+ */
+ if (!PointerIsAligned(p, uint16) &&
+ p + 1 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
+ p += 1;
+ }
+ if (!PointerIsAligned(p, uint32) &&
+ p + 2 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
+ p += 2;
+ }
+ if (!PointerIsAligned(p, uint64) &&
+ p + 4 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
+ p += 4;
+ }
+
+ /* Process eight bytes at a time, as far as we can. */
+ while (p + 8 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_d_w(*(uint64 *) p, crc);
+ p += 8;
+ }
+
+ /* Process remaining 0-7 bytes. */
+ if (p + 4 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
+ p += 4;
+ }
+ if (p + 2 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
+ p += 2;
+ }
+ if (p < pend)
+ {
+ crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
+ }
+
+ return crc;
+}
--
2.41.0
Attachments:
[text/plain] v2-0001-Add-loongarch-native-checksum-implementation.patch (14.1K, ../../[email protected]/2-v2-0001-Add-loongarch-native-checksum-implementation.patch)
download | inline diff:
From b2329478b5331e2aa9942c7ee4e23e3bfa871c1d Mon Sep 17 00:00:00 2001
From: YANG Xudong <[email protected]>
Date: Fri, 16 Jun 2023 09:22:08 +0800
Subject: [PATCH v2] Add loongarch native checksum implementation.
---
config/c-compiler.m4 | 34 ++++++++++++++++
configure | 73 ++++++++++++++++++++++++++++++----
configure.ac | 33 +++++++++++----
meson.build | 24 +++++++++++
src/include/pg_config.h.in | 3 ++
src/include/port/pg_crc32c.h | 9 +++++
src/port/meson.build | 3 ++
src/port/pg_crc32c_loongarch.c | 72 +++++++++++++++++++++++++++++++++
8 files changed, 236 insertions(+), 15 deletions(-)
create mode 100644 src/port/pg_crc32c_loongarch.c
diff --git a/config/c-compiler.m4 b/config/c-compiler.m4
index 5be8f0f08d..eb3af009c4 100644
--- a/config/c-compiler.m4
+++ b/config/c-compiler.m4
@@ -661,3 +661,37 @@ if test x"$Ac_cachevar" = x"yes"; then
fi
undefine([Ac_cachevar])dnl
])# PGAC_ARMV8_CRC32C_INTRINSICS
+
+# PGAC_LOONGARCH_CRC32C_INTRINSICS
+# ---------------------------
+# Check if the compiler supports the LoongArch CRCC instructions, using
+# __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w,
+# __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w
+# intrinsic functions.
+#
+# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics,
+# and CFLAGS_CRC.
+AC_DEFUN([PGAC_LOONGARCH_CRC32C_INTRINSICS],
+[define([Ac_cachevar], [AS_TR_SH([pgac_cv_loongarch_crc32c_intrinsics_$1])])dnl
+AC_CACHE_CHECK(
+ [for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=$1],
+ [Ac_cachevar],
+[pgac_save_CFLAGS=$CFLAGS
+CFLAGS="$pgac_save_CFLAGS $1"
+AC_LINK_IFELSE([AC_LANG_PROGRAM([],
+ [unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;])],
+ [Ac_cachevar=yes],
+ [Ac_cachevar=no])
+CFLAGS="$pgac_save_CFLAGS"])
+if test x"$Ac_cachevar" = x"yes"; then
+ CFLAGS_CRC="$1"
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+undefine([Ac_cachevar])dnl
+])# PGAC_LOONGARCH_CRC32C_INTRINSICS
diff --git a/configure b/configure
index 1b415142d1..7c60a26c11 100755
--- a/configure
+++ b/configure
@@ -18114,6 +18114,51 @@ fi
fi
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=" >&5
+$as_echo_n "checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w with CFLAGS=... " >&6; }
+if ${pgac_cv_loongarch_crc32c_intrinsics_+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ pgac_save_CFLAGS=$CFLAGS
+CFLAGS="$pgac_save_CFLAGS "
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+int
+main ()
+{
+unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ pgac_cv_loongarch_crc32c_intrinsics_=yes
+else
+ pgac_cv_loongarch_crc32c_intrinsics_=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+CFLAGS="$pgac_save_CFLAGS"
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_loongarch_crc32c_intrinsics_" >&5
+$as_echo "$pgac_cv_loongarch_crc32c_intrinsics_" >&6; }
+if test x"$pgac_cv_loongarch_crc32c_intrinsics_" = x"yes"; then
+ CFLAGS_CRC=""
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+
+
# Select CRC-32C implementation.
@@ -18132,7 +18177,7 @@ fi
#
# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
-if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x""; then
+if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
USE_SSE42_CRC32C=1
@@ -18150,10 +18195,15 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
- # fall back to slicing-by-8 algorithm, which doesn't require any
- # special CPU support.
- USE_SLICING_BY_8_CRC32C=1
- fi
+ if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
+ # LoongArch CRCC instructions.
+ USE_LOONGARCH_CRC32C=1
+ else
+ # fall back to slicing-by-8 algorithm, which doesn't require any
+ # special CPU support.
+ USE_SLICING_BY_8_CRC32C=1
+ fi
+ fi
fi
fi
fi
@@ -18194,12 +18244,21 @@ $as_echo "#define USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK 1" >>confdefs.h
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: ARMv8 CRC instructions with runtime check" >&5
$as_echo "ARMv8 CRC instructions with runtime check" >&6; }
else
+ if test x"$USE_LOONGARCH_CRC32C" = x"1"; then
+
+$as_echo "#define USE_LOONGARCH_CRC32C 1" >>confdefs.h
+
+ PG_CRC32C_OBJS="pg_crc32c_loongarch.o"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: LoongArch CRCC instructions" >&5
+$as_echo "LoongArch CRCC instructions" >&6; }
+ else
$as_echo "#define USE_SLICING_BY_8_CRC32C 1" >>confdefs.h
- PG_CRC32C_OBJS="pg_crc32c_sb8.o"
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: slicing-by-8" >&5
+ PG_CRC32C_OBJS="pg_crc32c_sb8.o"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: slicing-by-8" >&5
$as_echo "slicing-by-8" >&6; }
+ fi
fi
fi
fi
diff --git a/configure.ac b/configure.ac
index 09558ada0f..d44e9f9ef8 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2126,6 +2126,12 @@ if test x"$pgac_armv8_crc32c_intrinsics" != x"yes"; then
PGAC_ARMV8_CRC32C_INTRINSICS([-march=armv8-a+crc])
fi
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+PGAC_LOONGARCH_CRC32C_INTRINSICS([])
+
AC_SUBST(CFLAGS_CRC)
# Select CRC-32C implementation.
@@ -2144,7 +2150,7 @@ AC_SUBST(CFLAGS_CRC)
#
# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
-if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x""; then
+if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
USE_SSE42_CRC32C=1
@@ -2162,10 +2168,15 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
- # fall back to slicing-by-8 algorithm, which doesn't require any
- # special CPU support.
- USE_SLICING_BY_8_CRC32C=1
- fi
+ if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
+ # LoongArch CRCC instructions.
+ USE_LOONGARCH_CRC32C=1
+ else
+ # fall back to slicing-by-8 algorithm, which doesn't require any
+ # special CPU support.
+ USE_SLICING_BY_8_CRC32C=1
+ fi
+ fi
fi
fi
fi
@@ -2193,9 +2204,15 @@ else
PG_CRC32C_OBJS="pg_crc32c_armv8.o pg_crc32c_sb8.o pg_crc32c_armv8_choose.o"
AC_MSG_RESULT(ARMv8 CRC instructions with runtime check)
else
- AC_DEFINE(USE_SLICING_BY_8_CRC32C, 1, [Define to 1 to use software CRC-32C implementation (slicing-by-8).])
- PG_CRC32C_OBJS="pg_crc32c_sb8.o"
- AC_MSG_RESULT(slicing-by-8)
+ if test x"$USE_LOONGARCH_CRC32C" = x"1"; then
+ AC_DEFINE(USE_LOONGARCH_CRC32C, 1, [Define to 1 to use LoongArch CRCC instructions.])
+ PG_CRC32C_OBJS="pg_crc32c_loongarch.o"
+ AC_MSG_RESULT(LoongArch CRCC instructions)
+ else
+ AC_DEFINE(USE_SLICING_BY_8_CRC32C, 1, [Define to 1 to use software CRC-32C implementation (slicing-by-8).])
+ PG_CRC32C_OBJS="pg_crc32c_sb8.o"
+ AC_MSG_RESULT(slicing-by-8)
+ fi
fi
fi
fi
diff --git a/meson.build b/meson.build
index 82f2782673..77a9b985df 100644
--- a/meson.build
+++ b/meson.build
@@ -2073,6 +2073,30 @@ int main(void)
cdata.set('USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK', 1)
have_optimized_crc = true
endif
+
+elif host_cpu == 'loongarch64'
+
+ prog = '''
+int main(void)
+{
+ unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+}
+'''
+
+ if cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and __builtin_loongarch_crcc_w_d_w',
+ args: test_c_args)
+ # Use LoongArch CRC instruction unconditionally
+ cdata.set('USE_LOONGARCH_CRC32C', 1)
+ have_optimized_crc = true
+ endif
+
endif
if not have_optimized_crc
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 6d572c3820..1f253566e4 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -724,6 +724,9 @@
/* Define to 1 to build with LLVM based JIT support. (--with-llvm) */
#undef USE_LLVM
+/* Define to 1 to use LoongArch CRCC instructions. */
+#undef USE_LOONGARCH_CRC32C
+
/* Define to 1 to build with LZ4 support. (--with-lz4) */
#undef USE_LZ4
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 7f8779261c..d085f1dc00 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -58,6 +58,15 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+#elif defined(USE_LOONGARCH_CRC32C)
+/* Use LoongArch CRCC instructions. */
+
+#define COMP_CRC32C(crc, data, len) \
+ ((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
+#define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
+
+extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+
#elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
/*
diff --git a/src/port/meson.build b/src/port/meson.build
index 24416b9bfc..3e77b2493a 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -92,6 +92,9 @@ replace_funcs_pos = [
['pg_crc32c_armv8_choose', 'USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK'],
['pg_crc32c_sb8', 'USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK'],
+ # loongarch
+ ['pg_crc32c_loongarch', 'USE_LOONGARCH_CRC32C'],
+
# generic fallback
['pg_crc32c_sb8', 'USE_SLICING_BY_8_CRC32C'],
]
diff --git a/src/port/pg_crc32c_loongarch.c b/src/port/pg_crc32c_loongarch.c
new file mode 100644
index 0000000000..b54fe12ff5
--- /dev/null
+++ b/src/port/pg_crc32c_loongarch.c
@@ -0,0 +1,72 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_crc32c_loongarch.c
+ * Compute CRC-32C checksum using LoongArch CRCC instructions
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_crc32c_loongarch.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "c.h"
+
+#include "port/pg_crc32c.h"
+
+pg_crc32c
+pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len)
+{
+ const unsigned char *p = data;
+ const unsigned char *pend = p + len;
+
+ /*
+ * Aligned memory access is significantly faster.
+ * Process leading bytes so that the loop below starts with a pointer aligned to eight bytes.
+ */
+ if (!PointerIsAligned(p, uint16) &&
+ p + 1 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
+ p += 1;
+ }
+ if (!PointerIsAligned(p, uint32) &&
+ p + 2 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
+ p += 2;
+ }
+ if (!PointerIsAligned(p, uint64) &&
+ p + 4 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
+ p += 4;
+ }
+
+ /* Process eight bytes at a time, as far as we can. */
+ while (p + 8 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_d_w(*(uint64 *) p, crc);
+ p += 8;
+ }
+
+ /* Process remaining 0-7 bytes. */
+ if (p + 4 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
+ p += 4;
+ }
+ if (p + 2 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
+ p += 2;
+ }
+ if (p < pend)
+ {
+ crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
+ }
+
+ return crc;
+}
--
2.41.0
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
@ 2023-07-05 02:15 ` YANG Xudong <[email protected]>
2023-07-05 07:11 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-07-06 07:14 ` Re: Re: [PATCH] Add loongarch native checksum implementation. huchangqi <[email protected]>
1 sibling, 2 replies; 31+ messages in thread
From: YANG Xudong @ 2023-07-05 02:15 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]
Is there any other comment?
If the patch looks OK, I would like to update its status to ready for
committer in the commitfest.
Thanks!
On 2023/6/16 09:28, YANG Xudong wrote:
> Updated the patch based on the comments.
>
> On 2023/6/15 18:30, John Naylor wrote:
>>
>> On Wed, Jun 14, 2023 at 9:20 AM YANG Xudong <[email protected]
>> <mailto:[email protected]>> wrote:
>> >
>> > Attached a new patch with fixes based on the comment below.
>>
>> Note: It's helpful to pass "-v" to git format-patch, to have different
>> versions.
>>
>
> Added v2
>
>> > > For x86 and Arm, if it fails to link without an -march flag, we
>> allow
>> > > for a runtime check. The flags "-march=armv8-a+crc" and
>> "-msse4.2" are
>> > > for instructions not found on all platforms. The patch also
>> checks both
>> > > ways, and each one results in "Use LoongArch CRC instruction
>> > > unconditionally". The -march flag here is general, not specific. In
>> > > other words, if this only runs inside "+elif host_cpu ==
>> 'loongarch64'",
>> > > why do we need both with -march and without?
>> > >
>> >
>> > Removed the elif branch.
>>
>> Okay, since we've confirmed that no arch flag is necessary, some other
>> places can be simplified:
>>
>> --- a/src/port/Makefile
>> +++ b/src/port/Makefile
>> @@ -98,6 +98,11 @@ pg_crc32c_armv8.o: CFLAGS+=$(CFLAGS_CRC)
>> pg_crc32c_armv8_shlib.o: CFLAGS+=$(CFLAGS_CRC)
>> pg_crc32c_armv8_srv.o: CFLAGS+=$(CFLAGS_CRC)
>>
>> +# all versions of pg_crc32c_loongarch.o need CFLAGS_CRC
>> +pg_crc32c_loongarch.o: CFLAGS+=$(CFLAGS_CRC)
>> +pg_crc32c_loongarch_shlib.o: CFLAGS+=$(CFLAGS_CRC)
>> +pg_crc32c_loongarch_srv.o: CFLAGS+=$(CFLAGS_CRC)
>>
>> This was copy-and-pasted from platforms that use a runtime check, so
>> should be unnecessary.
>>
>
> Removed these lines.
>
>> +# If the intrinsics are supported, sets
>> pgac_loongarch_crc32c_intrinsics,
>> +# and CFLAGS_CRC.
>>
>> +# Check if __builtin_loongarch_crcc_* intrinsics can be used
>> +# with the default compiler flags.
>> +# CFLAGS_CRC is set if the extra flag is required.
>>
>> Same here -- it seems we don't need to set CFLAGS_CRC at all. Can you
>> confirm?
>>
>
> We don't need to set CFLAGS_CRC as commented. I have updated the
> configure script to make it align with the logic in meson build script.
>
>> > > Also, I don't have a Loongarch machine for testing. Could you
>> show that
>> > > the instructions are found in the binary, maybe using objdump and
>> grep?
>> > > Or a performance test?
>> > >
>> >
>> > The output of the objdump command `objdump -dS
>> > ../postgres-build/tmp_install/usr/local/pgsql/bin/postgres | grep
>> -B 30
>> > -A 10 crcc` is attached.
>>
>> Thanks for confirming.
>>
>> --
>> John Naylor
>> EDB: http://www.enterprisedb.com <http://www.enterprisedb.com;
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 02:15 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
@ 2023-07-05 07:11 ` John Naylor <[email protected]>
2023-07-26 03:16 ` Re: [PATCH] Add loongarch native checksum implementation. Michael Paquier <[email protected]>
1 sibling, 1 reply; 31+ messages in thread
From: John Naylor @ 2023-07-05 07:11 UTC (permalink / raw)
To: YANG Xudong <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]
On Wed, Jul 5, 2023 at 9:16 AM YANG Xudong <[email protected]> wrote:
>
> Is there any other comment?
It's only been a few weeks since the last patch, and this is not an urgent
bugfix, so there is no reason to ping the thread. Feature freeze will
likely be in April of next year.
Also, please don't top-post (which means: quoting an entire message, with
new text at the top) -- it clutters our archives.
Before I look at this again: Are there any objections to another CRC
implementation for the reason of having no buildfarm member?
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 02:15 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 07:11 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
@ 2023-07-26 03:16 ` Michael Paquier <[email protected]>
2023-07-26 04:37 ` Re: [PATCH] Add loongarch native checksum implementation. Nathan Bossart <[email protected]>
2023-07-26 05:36 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-26 07:38 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
0 siblings, 3 replies; 31+ messages in thread
From: Michael Paquier @ 2023-07-26 03:16 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: YANG Xudong <[email protected]>; [email protected]; [email protected]; [email protected]
On Wed, Jul 05, 2023 at 02:11:02PM +0700, John Naylor wrote:
> Also, please don't top-post (which means: quoting an entire message, with
> new text at the top) -- it clutters our archives.
>
> Before I look at this again: Are there any objections to another CRC
> implementation for the reason of having no buildfarm member?
The performance numbers presented upthread for the CRC computations
are kind of nice in this environment, but honestly I have no idea how
much this architecture is used. Perhaps that's only something in
China? I am not seeing much activity around that in Japan, for
instance, and that's really close.
Anyway, based on today's state of the buildfarm, we have a buildfarm
member named cisticola that should be able to test this new CRC
implementation, so I see no problem in applying this stuff now if you
think it is in good shape.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../ZMCQDL0wTy0%[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 02:15 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 07:11 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-07-26 03:16 ` Re: [PATCH] Add loongarch native checksum implementation. Michael Paquier <[email protected]>
@ 2023-07-26 04:37 ` Nathan Bossart <[email protected]>
2 siblings, 0 replies; 31+ messages in thread
From: Nathan Bossart @ 2023-07-26 04:37 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: John Naylor <[email protected]>; YANG Xudong <[email protected]>; [email protected]; [email protected]; [email protected]
On Wed, Jul 26, 2023 at 12:16:28PM +0900, Michael Paquier wrote:
> On Wed, Jul 05, 2023 at 02:11:02PM +0700, John Naylor wrote:
>> Before I look at this again: Are there any objections to another CRC
>> implementation for the reason of having no buildfarm member?
>
> [ ... ]
>
> Anyway, based on today's state of the buildfarm, we have a buildfarm
> member named cisticola that should be able to test this new CRC
> implementation, so I see no problem in applying this stuff now if you
> think it is in good shape.
IMHO we should strive to maintain buildfarm coverage for all the
instrinsics used within Postgres, if for no other reason than to ensure
future changes do not break those platforms.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 02:15 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 07:11 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-07-26 03:16 ` Re: [PATCH] Add loongarch native checksum implementation. Michael Paquier <[email protected]>
@ 2023-07-26 05:36 ` YANG Xudong <[email protected]>
2 siblings, 0 replies; 31+ messages in thread
From: YANG Xudong @ 2023-07-26 05:36 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]; John Naylor <[email protected]>; huchangqi <[email protected]>
On 2023/7/26 11:16, Michael Paquier wrote> The performance numbers
presented upthread for the CRC computations
> are kind of nice in this environment, but honestly I have no idea how
> much this architecture is used. Perhaps that's only something in
> China? I am not seeing much activity around that in Japan, for
> instance, and that's really close.
The architecture is pretty new (to open source ecosystem). The support
of it in Linux kernel and GCC were released last year.
Here is a site about the status of major open source project support of
it. (Chinese only)
https://areweloongyet.com/
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 02:15 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 07:11 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-07-26 03:16 ` Re: [PATCH] Add loongarch native checksum implementation. Michael Paquier <[email protected]>
@ 2023-07-26 07:38 ` John Naylor <[email protected]>
2 siblings, 0 replies; 31+ messages in thread
From: John Naylor @ 2023-07-26 07:38 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: YANG Xudong <[email protected]>; [email protected]; [email protected]; [email protected]
On Wed, Jul 26, 2023 at 8:25 AM YANG Xudong <[email protected]> wrote:
>
> Many thanks to huchangqi. Now we have loongarch64 support for both old
> world ABI and new world ABI on the buildfarm!
Glad to hear it!
On Wed, Jul 26, 2023 at 10:16 AM Michael Paquier <[email protected]>
wrote:
>
> The performance numbers presented upthread for the CRC computations
> are kind of nice in this environment, but honestly I have no idea how
> much this architecture is used. Perhaps that's only something in
> China? I am not seeing much activity around that in Japan, for
> instance, and that's really close.
That was my impression as well. My thinking was, we can give the same
treatment that we gave Arm a number of years ago (which is now quite
mainstream).
> Anyway, based on today's state of the buildfarm, we have a buildfarm
> member named cisticola that should be able to test this new CRC
> implementation, so I see no problem in applying this stuff now if you
> think it is in good shape.
I believe there was just a comment that needed updating, so I'll do that
and push within a few days.
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 02:15 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
@ 2023-07-06 07:14 ` huchangqi <[email protected]>
2023-07-06 09:48 ` Re: [PATCH] Add loongarch native checksum implementation. Daniel Gustafsson <[email protected]>
2023-07-06 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
1 sibling, 2 replies; 31+ messages in thread
From: huchangqi @ 2023-07-06 07:14 UTC (permalink / raw)
To: YANG Xudong <[email protected]>; +Cc: John Naylor <[email protected]>; [email protected]; [email protected]; [email protected]
Hi, i have a loongarch machine runing Loongnix-server (based on redhat/centos, it has gcc-8.3 on it), i am trying to test buildfarm on it, when i edit build-farm.conf it seems to need animal and secret to connect the buildfarm server.
then i go to https://buildfarm.postgresql.org/cgi-bin/register-form.pl, and registered the buildfarm a week ago, but didn't receive any response. so what else i need to do next.
> -----Original Messages-----
> From: "YANG Xudong" <[email protected]>
> Send time:Wednesday, 07/05/2023 10:15:51
> To: "John Naylor" <[email protected]>
> Cc: [email protected], [email protected], [email protected]
> Subject: Re: [PATCH] Add loongarch native checksum implementation.
>
> Is there any other comment?
>
> If the patch looks OK, I would like to update its status to ready for
> committer in the commitfest.
>
> Thanks!
>
> On 2023/6/16 09:28, YANG Xudong wrote:
> > Updated the patch based on the comments.
> >
> > On 2023/6/15 18:30, John Naylor wrote:
> >>
> >> On Wed, Jun 14, 2023 at 9:20 AM YANG Xudong <[email protected]
> >> <mailto:[email protected]>> wrote:
> >> >
> >> > Attached a new patch with fixes based on the comment below.
> >>
> >> Note: It's helpful to pass "-v" to git format-patch, to have different
> >> versions.
> >>
> >
> > Added v2
> >
> >> > > For x86 and Arm, if it fails to link without an -march flag, we
> >> allow
> >> > > for a runtime check. The flags "-march=armv8-a+crc" and
> >> "-msse4.2" are
> >> > > for instructions not found on all platforms. The patch also
> >> checks both
> >> > > ways, and each one results in "Use LoongArch CRC instruction
> >> > > unconditionally". The -march flag here is general, not specific. In
> >> > > other words, if this only runs inside "+elif host_cpu ==
> >> 'loongarch64'",
> >> > > why do we need both with -march and without?
> >> > >
> >> >
> >> > Removed the elif branch.
> >>
> >> Okay, since we've confirmed that no arch flag is necessary, some other
> >> places can be simplified:
> >>
> >> --- a/src/port/Makefile
> >> +++ b/src/port/Makefile
> >> @@ -98,6 +98,11 @@ pg_crc32c_armv8.o: CFLAGS+=$(CFLAGS_CRC)
> >> pg_crc32c_armv8_shlib.o: CFLAGS+=$(CFLAGS_CRC)
> >> pg_crc32c_armv8_srv.o: CFLAGS+=$(CFLAGS_CRC)
> >>
> >> +# all versions of pg_crc32c_loongarch.o need CFLAGS_CRC
> >> +pg_crc32c_loongarch.o: CFLAGS+=$(CFLAGS_CRC)
> >> +pg_crc32c_loongarch_shlib.o: CFLAGS+=$(CFLAGS_CRC)
> >> +pg_crc32c_loongarch_srv.o: CFLAGS+=$(CFLAGS_CRC)
> >>
> >> This was copy-and-pasted from platforms that use a runtime check, so
> >> should be unnecessary.
> >>
> >
> > Removed these lines.
> >
> >> +# If the intrinsics are supported, sets
> >> pgac_loongarch_crc32c_intrinsics,
> >> +# and CFLAGS_CRC.
> >>
> >> +# Check if __builtin_loongarch_crcc_* intrinsics can be used
> >> +# with the default compiler flags.
> >> +# CFLAGS_CRC is set if the extra flag is required.
> >>
> >> Same here -- it seems we don't need to set CFLAGS_CRC at all. Can you
> >> confirm?
> >>
> >
> > We don't need to set CFLAGS_CRC as commented. I have updated the
> > configure script to make it align with the logic in meson build script.
> >
> >> > > Also, I don't have a Loongarch machine for testing. Could you
> >> show that
> >> > > the instructions are found in the binary, maybe using objdump and
> >> grep?
> >> > > Or a performance test?
> >> > >
> >> >
> >> > The output of the objdump command `objdump -dS
> >> > ../postgres-build/tmp_install/usr/local/pgsql/bin/postgres | grep
> >> -B 30
> >> > -A 10 crcc` is attached.
> >>
> >> Thanks for confirming.
> >>
> >> --
> >> John Naylor
> >> EDB: http://www.enterprisedb.com <http://www.enterprisedb.com;
>
本邮件及其附件含有龙芯中科的商业秘密信息，仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制或散发）本邮件及其附件中的信息。如果您错收本邮件，请您立即电话或邮件通知发件人并删除本邮件。
This email and its attachments contain confidential information from Loongson Technology , which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or email immediately and delete it.
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 02:15 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-06 07:14 ` Re: Re: [PATCH] Add loongarch native checksum implementation. huchangqi <[email protected]>
@ 2023-07-06 09:48 ` Daniel Gustafsson <[email protected]>
1 sibling, 0 replies; 31+ messages in thread
From: Daniel Gustafsson @ 2023-07-06 09:48 UTC (permalink / raw)
To: huchangqi <[email protected]>; +Cc: YANG Xudong <[email protected]>; John Naylor <[email protected]>; PostgreSQL Hackers <[email protected]>; [email protected]; [email protected]; Andrew Dunstan <[email protected]>
> On 6 Jul 2023, at 09:14, huchangqi <[email protected]> wrote:
>
> Hi, i have a loongarch machine runing Loongnix-server (based on redhat/centos, it has gcc-8.3 on it), i am trying to test buildfarm on it, when i edit build-farm.conf it seems to need animal and secret to connect the buildfarm server.
> then i go to https://buildfarm.postgresql.org/cgi-bin/register-form.pl, and registered the buildfarm a week ago, but didn't receive any response. so what else i need to do next.
Thanks for volunteering a buildfarm animal! The registration is probably just
pending due to it being summer and things slow down, but I've added Andrew
Dunstan who is the Buildfarm expert on CC:.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 02:15 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-06 07:14 ` Re: Re: [PATCH] Add loongarch native checksum implementation. huchangqi <[email protected]>
@ 2023-07-06 10:30 ` YANG Xudong <[email protected]>
2023-07-26 01:25 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
1 sibling, 1 reply; 31+ messages in thread
From: YANG Xudong @ 2023-07-06 10:30 UTC (permalink / raw)
To: huchangqi <[email protected]>; +Cc: John Naylor <[email protected]>; [email protected]; [email protected]; [email protected]
On 2023/7/6 15:14, huchangqi wrote:
> Hi, i have a loongarch machine runing Loongnix-server (based on redhat/centos, it has gcc-8.3 on it), i am trying to test buildfarm on it, when i edit build-farm.conf it seems to need animal and secret to connect the buildfarm server.
> then i go to https://buildfarm.postgresql.org/cgi-bin/register-form.pl, and registered the buildfarm a week ago, but didn't receive any response. so what else i need to do next.
>
>
Is it possible to provide a build farm instance for new world ABI of
loongarch also by loongson? It will be really appreciated.
Thanks!
>
>> -----Original Messages-----
>> From: "YANG Xudong" <[email protected]>
>> Send time:Wednesday, 07/05/2023 10:15:51
>> To: "John Naylor" <[email protected]>
>> Cc: [email protected], [email protected], [email protected]
>> Subject: Re: [PATCH] Add loongarch native checksum implementation.
>>
>> Is there any other comment?
>>
>> If the patch looks OK, I would like to update its status to ready for
>> committer in the commitfest.
>>
>> Thanks!
>>
>> On 2023/6/16 09:28, YANG Xudong wrote:
>>> Updated the patch based on the comments.
>>>
>>> On 2023/6/15 18:30, John Naylor wrote:
>>>>
>>>> On Wed, Jun 14, 2023 at 9:20 AM YANG Xudong <[email protected]
>>>> <mailto:[email protected]>> wrote:
>>>> >
>>>> > Attached a new patch with fixes based on the comment below.
>>>>
>>>> Note: It's helpful to pass "-v" to git format-patch, to have different
>>>> versions.
>>>>
>>>
>>> Added v2
>>>
>>>> > > For x86 and Arm, if it fails to link without an -march flag, we
>>>> allow
>>>> > > for a runtime check. The flags "-march=armv8-a+crc" and
>>>> "-msse4.2" are
>>>> > > for instructions not found on all platforms. The patch also
>>>> checks both
>>>> > > ways, and each one results in "Use LoongArch CRC instruction
>>>> > > unconditionally". The -march flag here is general, not specific. In
>>>> > > other words, if this only runs inside "+elif host_cpu ==
>>>> 'loongarch64'",
>>>> > > why do we need both with -march and without?
>>>> > >
>>>> >
>>>> > Removed the elif branch.
>>>>
>>>> Okay, since we've confirmed that no arch flag is necessary, some other
>>>> places can be simplified:
>>>>
>>>> --- a/src/port/Makefile
>>>> +++ b/src/port/Makefile
>>>> @@ -98,6 +98,11 @@ pg_crc32c_armv8.o: CFLAGS+=$(CFLAGS_CRC)
>>>> pg_crc32c_armv8_shlib.o: CFLAGS+=$(CFLAGS_CRC)
>>>> pg_crc32c_armv8_srv.o: CFLAGS+=$(CFLAGS_CRC)
>>>>
>>>> +# all versions of pg_crc32c_loongarch.o need CFLAGS_CRC
>>>> +pg_crc32c_loongarch.o: CFLAGS+=$(CFLAGS_CRC)
>>>> +pg_crc32c_loongarch_shlib.o: CFLAGS+=$(CFLAGS_CRC)
>>>> +pg_crc32c_loongarch_srv.o: CFLAGS+=$(CFLAGS_CRC)
>>>>
>>>> This was copy-and-pasted from platforms that use a runtime check, so
>>>> should be unnecessary.
>>>>
>>>
>>> Removed these lines.
>>>
>>>> +# If the intrinsics are supported, sets
>>>> pgac_loongarch_crc32c_intrinsics,
>>>> +# and CFLAGS_CRC.
>>>>
>>>> +# Check if __builtin_loongarch_crcc_* intrinsics can be used
>>>> +# with the default compiler flags.
>>>> +# CFLAGS_CRC is set if the extra flag is required.
>>>>
>>>> Same here -- it seems we don't need to set CFLAGS_CRC at all. Can you
>>>> confirm?
>>>>
>>>
>>> We don't need to set CFLAGS_CRC as commented. I have updated the
>>> configure script to make it align with the logic in meson build script.
>>>
>>>> > > Also, I don't have a Loongarch machine for testing. Could you
>>>> show that
>>>> > > the instructions are found in the binary, maybe using objdump and
>>>> grep?
>>>> > > Or a performance test?
>>>> > >
>>>> >
>>>> > The output of the objdump command `objdump -dS
>>>> > ../postgres-build/tmp_install/usr/local/pgsql/bin/postgres | grep
>>>> -B 30
>>>> > -A 10 crcc` is attached.
>>>>
>>>> Thanks for confirming.
>>>>
>>>> --
>>>> John Naylor
>>>> EDB: http://www.enterprisedb.com <http://www.enterprisedb.com;
>>
>
>
> 本邮件及其附件含有龙芯中科的商业秘密信息，仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制或散发）本邮件及其附件中的信息。如果您错收本邮件，请您立即电话或邮件通知发件人并删除本邮件。
> This email and its attachments contain confidential information from Loongson Technology , which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or email immediately and delete it.
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 02:15 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-06 07:14 ` Re: Re: [PATCH] Add loongarch native checksum implementation. huchangqi <[email protected]>
2023-07-06 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
@ 2023-07-26 01:25 ` YANG Xudong <[email protected]>
0 siblings, 0 replies; 31+ messages in thread
From: YANG Xudong @ 2023-07-26 01:25 UTC (permalink / raw)
To: huchangqi <[email protected]>; John Naylor <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]
Many thanks to huchangqi. Now we have loongarch64 support for both old
world ABI and new world ABI on the buildfarm!
-------- Forwarded Message --------
Subject: Re: [PATCH] Add loongarch native checksum implementation.
Date: Tue, 25 Jul 2023 15:51:43 +0800
From: huchangqi <[email protected]>
To: YANG Xudong <[email protected]>
Both cisticola and nuthatch are on the buildfarm now。
cisticola is "old world ABI".
https://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=cisticola&br=HEAD
nuthatch is "new world ABI".
https://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=nuthatch&br=HEAD
----------
Best regards,
huchangqi
On 2023/7/5 15:11, John Naylor wrote:
>
> Before I look at this again: Are there any objections to another CRC
> implementation for the reason of having no buildfarm member?
It is possible to try this patch on buildfarm now, I guess?
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
@ 2023-08-07 11:01 ` John Naylor <[email protected]>
2023-08-08 03:06 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
1 sibling, 1 reply; 31+ messages in thread
From: John Naylor @ 2023-08-07 11:01 UTC (permalink / raw)
To: YANG Xudong <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]
On Fri, Jun 16, 2023 at 8:28 AM YANG Xudong <[email protected]> wrote:
> > +# If the intrinsics are supported, sets
pgac_loongarch_crc32c_intrinsics,
> > +# and CFLAGS_CRC.
> >
> > +# Check if __builtin_loongarch_crcc_* intrinsics can be used
> > +# with the default compiler flags.
> > +# CFLAGS_CRC is set if the extra flag is required.
> >
> > Same here -- it seems we don't need to set CFLAGS_CRC at all. Can you
> > confirm?
> >
>
> We don't need to set CFLAGS_CRC as commented. I have updated the
> configure script to make it align with the logic in meson build script.
(Looking again at v2)
The compilation test is found in c-compiler.m4, which still has all logic
for CFLAGS_CRC, including saving and restoring the old CFLAGS. Can this
also be simplified?
I diff'd pg_crc32c_loongarch.c with the current other files, and found it
is structurally the same as the Arm implementation. That's logical if
memory alignment is important.
/*
- * ARMv8 doesn't require alignment, but aligned memory access is
- * significantly faster. Process leading bytes so that the loop below
- * starts with a pointer aligned to eight bytes.
+ * Aligned memory access is significantly faster.
+ * Process leading bytes so that the loop below starts with a pointer
aligned to eight bytes.
Can you confirm the alignment requirement -- it's not clear what the
intention is since "doesn't require" wasn't carried over. Is there any
documentation (or even a report in some other context) about aligned vs
unaligned memory access performance?
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-07 11:01 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
@ 2023-08-08 03:06 ` YANG Xudong <[email protected]>
2023-08-08 06:38 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: YANG Xudong @ 2023-08-08 03:06 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]
Thanks for the comment. I have updated the patch to v3. Please have a look.
On 2023/8/7 19:01, John Naylor wrote:
>
> On Fri, Jun 16, 2023 at 8:28 AM YANG Xudong <[email protected]
> <mailto:[email protected]>> wrote:
> > > +# If the intrinsics are supported, sets
> pgac_loongarch_crc32c_intrinsics,
> > > +# and CFLAGS_CRC.
> > >
> > > +# Check if __builtin_loongarch_crcc_* intrinsics can be used
> > > +# with the default compiler flags.
> > > +# CFLAGS_CRC is set if the extra flag is required.
> > >
> > > Same here -- it seems we don't need to set CFLAGS_CRC at all. Can you
> > > confirm?
> > >
> >
> > We don't need to set CFLAGS_CRC as commented. I have updated the
> > configure script to make it align with the logic in meson build script.
>
> (Looking again at v2)
>
> The compilation test is found in c-compiler.m4, which still has all
> logic for CFLAGS_CRC, including saving and restoring the old CFLAGS. Can
> this also be simplified?
Fixed the function in c-compiler.m4 by removing the function argument
and the logic of handling CFLAGS and CFLAGS_CRC.
>
> I diff'd pg_crc32c_loongarch.c with the current other files, and found
> it is structurally the same as the Arm implementation. That's logical if
> memory alignment is important.
>
> /*
> - * ARMv8 doesn't require alignment, but aligned memory access is
> - * significantly faster. Process leading bytes so that the loop below
> - * starts with a pointer aligned to eight bytes.
> + * Aligned memory access is significantly faster.
> + * Process leading bytes so that the loop below starts with a pointer
> aligned to eight bytes.
>
> Can you confirm the alignment requirement -- it's not clear what the
> intention is since "doesn't require" wasn't carried over. Is there any
> documentation (or even a report in some other context) about aligned vs
> unaligned memory access performance?
It is in the official document that the alignment is not required.
https://github.com/loongson/la-softdev-convention/blob/master/la-softdev-convention.adoc#74-unaligne...
However, I found this patch in LKML that shows great performance gain
when using aligned memory access similar to this patch.
https://lore.kernel.org/lkml/[email protected]/
So I guess using aligned memory access is necessary and I have updated
the comment in the code.
>
> --
> John Naylor
> EDB: http://www.enterprisedb.com <http://www.enterprisedb.com;
From ced3f65d7445bdcca0628c4f5073b5657a81cd28 Mon Sep 17 00:00:00 2001
From: YANG Xudong <[email protected]>
Date: Tue, 8 Aug 2023 10:41:58 +0800
Subject: [PATCH v3] Add loongarch native checksum implementation.
---
config/c-compiler.m4 | 29 ++++++++++++++
configure | 69 ++++++++++++++++++++++++++++----
configure.ac | 33 +++++++++++----
meson.build | 24 +++++++++++
src/include/pg_config.h.in | 3 ++
src/include/port/pg_crc32c.h | 9 +++++
src/port/meson.build | 3 ++
src/port/pg_crc32c_loongarch.c | 73 ++++++++++++++++++++++++++++++++++
8 files changed, 228 insertions(+), 15 deletions(-)
create mode 100644 src/port/pg_crc32c_loongarch.c
diff --git a/config/c-compiler.m4 b/config/c-compiler.m4
index 5be8f0f08d..7777ad6e90 100644
--- a/config/c-compiler.m4
+++ b/config/c-compiler.m4
@@ -661,3 +661,32 @@ if test x"$Ac_cachevar" = x"yes"; then
fi
undefine([Ac_cachevar])dnl
])# PGAC_ARMV8_CRC32C_INTRINSICS
+
+# PGAC_LOONGARCH_CRC32C_INTRINSICS
+# ---------------------------
+# Check if the compiler supports the LoongArch CRCC instructions, using
+# __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w,
+# __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w
+# intrinsic functions.
+#
+# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics.
+AC_DEFUN([PGAC_LOONGARCH_CRC32C_INTRINSICS],
+[define([Ac_cachevar], [AS_TR_SH([pgac_cv_loongarch_crc32c_intrinsics])])dnl
+AC_CACHE_CHECK(
+ [for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w],
+ [Ac_cachevar],
+[AC_LINK_IFELSE([AC_LANG_PROGRAM([],
+ [unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;])],
+ [Ac_cachevar=yes],
+ [Ac_cachevar=no])])
+if test x"$Ac_cachevar" = x"yes"; then
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+undefine([Ac_cachevar])dnl
+])# PGAC_LOONGARCH_CRC32C_INTRINSICS
diff --git a/configure b/configure
index 963fbbcf1e..c5eb2814f1 100755
--- a/configure
+++ b/configure
@@ -18047,6 +18047,47 @@ fi
fi
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w" >&5
+$as_echo_n "checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w... " >&6; }
+if ${pgac_cv_loongarch_crc32c_intrinsics+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+int
+main ()
+{
+unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ pgac_cv_loongarch_crc32c_intrinsics=yes
+else
+ pgac_cv_loongarch_crc32c_intrinsics=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_loongarch_crc32c_intrinsics" >&5
+$as_echo "$pgac_cv_loongarch_crc32c_intrinsics" >&6; }
+if test x"$pgac_cv_loongarch_crc32c_intrinsics" = x"yes"; then
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+
+
# Select CRC-32C implementation.
@@ -18065,7 +18106,7 @@ fi
#
# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
-if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x""; then
+if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
USE_SSE42_CRC32C=1
@@ -18083,10 +18124,15 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
- # fall back to slicing-by-8 algorithm, which doesn't require any
- # special CPU support.
- USE_SLICING_BY_8_CRC32C=1
- fi
+ if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
+ # LoongArch CRCC instructions.
+ USE_LOONGARCH_CRC32C=1
+ else
+ # fall back to slicing-by-8 algorithm, which doesn't require any
+ # special CPU support.
+ USE_SLICING_BY_8_CRC32C=1
+ fi
+ fi
fi
fi
fi
@@ -18127,12 +18173,21 @@ $as_echo "#define USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK 1" >>confdefs.h
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: ARMv8 CRC instructions with runtime check" >&5
$as_echo "ARMv8 CRC instructions with runtime check" >&6; }
else
+ if test x"$USE_LOONGARCH_CRC32C" = x"1"; then
+
+$as_echo "#define USE_LOONGARCH_CRC32C 1" >>confdefs.h
+
+ PG_CRC32C_OBJS="pg_crc32c_loongarch.o"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: LoongArch CRCC instructions" >&5
+$as_echo "LoongArch CRCC instructions" >&6; }
+ else
$as_echo "#define USE_SLICING_BY_8_CRC32C 1" >>confdefs.h
- PG_CRC32C_OBJS="pg_crc32c_sb8.o"
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: slicing-by-8" >&5
+ PG_CRC32C_OBJS="pg_crc32c_sb8.o"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: slicing-by-8" >&5
$as_echo "slicing-by-8" >&6; }
+ fi
fi
fi
fi
diff --git a/configure.ac b/configure.ac
index 5153b8b3fd..3011b8ea34 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2099,6 +2099,12 @@ if test x"$pgac_armv8_crc32c_intrinsics" != x"yes"; then
PGAC_ARMV8_CRC32C_INTRINSICS([-march=armv8-a+crc])
fi
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+PGAC_LOONGARCH_CRC32C_INTRINSICS()
+
AC_SUBST(CFLAGS_CRC)
# Select CRC-32C implementation.
@@ -2117,7 +2123,7 @@ AC_SUBST(CFLAGS_CRC)
#
# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
-if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x""; then
+if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
USE_SSE42_CRC32C=1
@@ -2135,10 +2141,15 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
- # fall back to slicing-by-8 algorithm, which doesn't require any
- # special CPU support.
- USE_SLICING_BY_8_CRC32C=1
- fi
+ if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
+ # LoongArch CRCC instructions.
+ USE_LOONGARCH_CRC32C=1
+ else
+ # fall back to slicing-by-8 algorithm, which doesn't require any
+ # special CPU support.
+ USE_SLICING_BY_8_CRC32C=1
+ fi
+ fi
fi
fi
fi
@@ -2166,9 +2177,15 @@ else
PG_CRC32C_OBJS="pg_crc32c_armv8.o pg_crc32c_sb8.o pg_crc32c_armv8_choose.o"
AC_MSG_RESULT(ARMv8 CRC instructions with runtime check)
else
- AC_DEFINE(USE_SLICING_BY_8_CRC32C, 1, [Define to 1 to use software CRC-32C implementation (slicing-by-8).])
- PG_CRC32C_OBJS="pg_crc32c_sb8.o"
- AC_MSG_RESULT(slicing-by-8)
+ if test x"$USE_LOONGARCH_CRC32C" = x"1"; then
+ AC_DEFINE(USE_LOONGARCH_CRC32C, 1, [Define to 1 to use LoongArch CRCC instructions.])
+ PG_CRC32C_OBJS="pg_crc32c_loongarch.o"
+ AC_MSG_RESULT(LoongArch CRCC instructions)
+ else
+ AC_DEFINE(USE_SLICING_BY_8_CRC32C, 1, [Define to 1 to use software CRC-32C implementation (slicing-by-8).])
+ PG_CRC32C_OBJS="pg_crc32c_sb8.o"
+ AC_MSG_RESULT(slicing-by-8)
+ fi
fi
fi
fi
diff --git a/meson.build b/meson.build
index 0a11efc97a..2acb204003 100644
--- a/meson.build
+++ b/meson.build
@@ -2065,6 +2065,30 @@ int main(void)
cdata.set('USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK', 1)
have_optimized_crc = true
endif
+
+elif host_cpu == 'loongarch64'
+
+ prog = '''
+int main(void)
+{
+ unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+}
+'''
+
+ if cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and __builtin_loongarch_crcc_w_d_w',
+ args: test_c_args)
+ # Use LoongArch CRC instruction unconditionally
+ cdata.set('USE_LOONGARCH_CRC32C', 1)
+ have_optimized_crc = true
+ endif
+
endif
if not have_optimized_crc
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index ee209d6d70..d8a2985567 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -714,6 +714,9 @@
/* Define to 1 to build with LLVM based JIT support. (--with-llvm) */
#undef USE_LLVM
+/* Define to 1 to use LoongArch CRCC instructions. */
+#undef USE_LOONGARCH_CRC32C
+
/* Define to 1 to build with LZ4 support. (--with-lz4) */
#undef USE_LZ4
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 7f8779261c..d085f1dc00 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -58,6 +58,15 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+#elif defined(USE_LOONGARCH_CRC32C)
+/* Use LoongArch CRCC instructions. */
+
+#define COMP_CRC32C(crc, data, len) \
+ ((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
+#define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
+
+extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+
#elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
/*
diff --git a/src/port/meson.build b/src/port/meson.build
index 9d0cd93c43..deb354418d 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -92,6 +92,9 @@ replace_funcs_pos = [
['pg_crc32c_armv8_choose', 'USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK'],
['pg_crc32c_sb8', 'USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK'],
+ # loongarch
+ ['pg_crc32c_loongarch', 'USE_LOONGARCH_CRC32C'],
+
# generic fallback
['pg_crc32c_sb8', 'USE_SLICING_BY_8_CRC32C'],
]
diff --git a/src/port/pg_crc32c_loongarch.c b/src/port/pg_crc32c_loongarch.c
new file mode 100644
index 0000000000..2897920800
--- /dev/null
+++ b/src/port/pg_crc32c_loongarch.c
@@ -0,0 +1,73 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_crc32c_loongarch.c
+ * Compute CRC-32C checksum using LoongArch CRCC instructions
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_crc32c_loongarch.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "c.h"
+
+#include "port/pg_crc32c.h"
+
+pg_crc32c
+pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len)
+{
+ const unsigned char *p = data;
+ const unsigned char *pend = p + len;
+
+ /*
+ * Loongarch desktop and server chips support unaligned memory access by default.
+ * However, aligned memory access is significantly faster.
+ * Process leading bytes so that the loop below starts with a pointer aligned to eight bytes.
+ */
+ if (!PointerIsAligned(p, uint16) &&
+ p + 1 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
+ p += 1;
+ }
+ if (!PointerIsAligned(p, uint32) &&
+ p + 2 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
+ p += 2;
+ }
+ if (!PointerIsAligned(p, uint64) &&
+ p + 4 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
+ p += 4;
+ }
+
+ /* Process eight bytes at a time, as far as we can. */
+ while (p + 8 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_d_w(*(uint64 *) p, crc);
+ p += 8;
+ }
+
+ /* Process remaining 0-7 bytes. */
+ if (p + 4 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
+ p += 4;
+ }
+ if (p + 2 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
+ p += 2;
+ }
+ if (p < pend)
+ {
+ crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
+ }
+
+ return crc;
+}
--
2.41.0
Attachments:
[text/plain] v3-0001-Add-loongarch-native-checksum-implementation.patch (13.9K, ../../[email protected]/2-v3-0001-Add-loongarch-native-checksum-implementation.patch)
download | inline diff:
From ced3f65d7445bdcca0628c4f5073b5657a81cd28 Mon Sep 17 00:00:00 2001
From: YANG Xudong <[email protected]>
Date: Tue, 8 Aug 2023 10:41:58 +0800
Subject: [PATCH v3] Add loongarch native checksum implementation.
---
config/c-compiler.m4 | 29 ++++++++++++++
configure | 69 ++++++++++++++++++++++++++++----
configure.ac | 33 +++++++++++----
meson.build | 24 +++++++++++
src/include/pg_config.h.in | 3 ++
src/include/port/pg_crc32c.h | 9 +++++
src/port/meson.build | 3 ++
src/port/pg_crc32c_loongarch.c | 73 ++++++++++++++++++++++++++++++++++
8 files changed, 228 insertions(+), 15 deletions(-)
create mode 100644 src/port/pg_crc32c_loongarch.c
diff --git a/config/c-compiler.m4 b/config/c-compiler.m4
index 5be8f0f08d..7777ad6e90 100644
--- a/config/c-compiler.m4
+++ b/config/c-compiler.m4
@@ -661,3 +661,32 @@ if test x"$Ac_cachevar" = x"yes"; then
fi
undefine([Ac_cachevar])dnl
])# PGAC_ARMV8_CRC32C_INTRINSICS
+
+# PGAC_LOONGARCH_CRC32C_INTRINSICS
+# ---------------------------
+# Check if the compiler supports the LoongArch CRCC instructions, using
+# __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w,
+# __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w
+# intrinsic functions.
+#
+# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics.
+AC_DEFUN([PGAC_LOONGARCH_CRC32C_INTRINSICS],
+[define([Ac_cachevar], [AS_TR_SH([pgac_cv_loongarch_crc32c_intrinsics])])dnl
+AC_CACHE_CHECK(
+ [for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w],
+ [Ac_cachevar],
+[AC_LINK_IFELSE([AC_LANG_PROGRAM([],
+ [unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;])],
+ [Ac_cachevar=yes],
+ [Ac_cachevar=no])])
+if test x"$Ac_cachevar" = x"yes"; then
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+undefine([Ac_cachevar])dnl
+])# PGAC_LOONGARCH_CRC32C_INTRINSICS
diff --git a/configure b/configure
index 963fbbcf1e..c5eb2814f1 100755
--- a/configure
+++ b/configure
@@ -18047,6 +18047,47 @@ fi
fi
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w" >&5
+$as_echo_n "checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w... " >&6; }
+if ${pgac_cv_loongarch_crc32c_intrinsics+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+int
+main ()
+{
+unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ pgac_cv_loongarch_crc32c_intrinsics=yes
+else
+ pgac_cv_loongarch_crc32c_intrinsics=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_loongarch_crc32c_intrinsics" >&5
+$as_echo "$pgac_cv_loongarch_crc32c_intrinsics" >&6; }
+if test x"$pgac_cv_loongarch_crc32c_intrinsics" = x"yes"; then
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+
+
# Select CRC-32C implementation.
@@ -18065,7 +18106,7 @@ fi
#
# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
-if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x""; then
+if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
USE_SSE42_CRC32C=1
@@ -18083,10 +18124,15 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
- # fall back to slicing-by-8 algorithm, which doesn't require any
- # special CPU support.
- USE_SLICING_BY_8_CRC32C=1
- fi
+ if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
+ # LoongArch CRCC instructions.
+ USE_LOONGARCH_CRC32C=1
+ else
+ # fall back to slicing-by-8 algorithm, which doesn't require any
+ # special CPU support.
+ USE_SLICING_BY_8_CRC32C=1
+ fi
+ fi
fi
fi
fi
@@ -18127,12 +18173,21 @@ $as_echo "#define USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK 1" >>confdefs.h
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: ARMv8 CRC instructions with runtime check" >&5
$as_echo "ARMv8 CRC instructions with runtime check" >&6; }
else
+ if test x"$USE_LOONGARCH_CRC32C" = x"1"; then
+
+$as_echo "#define USE_LOONGARCH_CRC32C 1" >>confdefs.h
+
+ PG_CRC32C_OBJS="pg_crc32c_loongarch.o"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: LoongArch CRCC instructions" >&5
+$as_echo "LoongArch CRCC instructions" >&6; }
+ else
$as_echo "#define USE_SLICING_BY_8_CRC32C 1" >>confdefs.h
- PG_CRC32C_OBJS="pg_crc32c_sb8.o"
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: slicing-by-8" >&5
+ PG_CRC32C_OBJS="pg_crc32c_sb8.o"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: slicing-by-8" >&5
$as_echo "slicing-by-8" >&6; }
+ fi
fi
fi
fi
diff --git a/configure.ac b/configure.ac
index 5153b8b3fd..3011b8ea34 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2099,6 +2099,12 @@ if test x"$pgac_armv8_crc32c_intrinsics" != x"yes"; then
PGAC_ARMV8_CRC32C_INTRINSICS([-march=armv8-a+crc])
fi
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+PGAC_LOONGARCH_CRC32C_INTRINSICS()
+
AC_SUBST(CFLAGS_CRC)
# Select CRC-32C implementation.
@@ -2117,7 +2123,7 @@ AC_SUBST(CFLAGS_CRC)
#
# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
-if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x""; then
+if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
USE_SSE42_CRC32C=1
@@ -2135,10 +2141,15 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
- # fall back to slicing-by-8 algorithm, which doesn't require any
- # special CPU support.
- USE_SLICING_BY_8_CRC32C=1
- fi
+ if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
+ # LoongArch CRCC instructions.
+ USE_LOONGARCH_CRC32C=1
+ else
+ # fall back to slicing-by-8 algorithm, which doesn't require any
+ # special CPU support.
+ USE_SLICING_BY_8_CRC32C=1
+ fi
+ fi
fi
fi
fi
@@ -2166,9 +2177,15 @@ else
PG_CRC32C_OBJS="pg_crc32c_armv8.o pg_crc32c_sb8.o pg_crc32c_armv8_choose.o"
AC_MSG_RESULT(ARMv8 CRC instructions with runtime check)
else
- AC_DEFINE(USE_SLICING_BY_8_CRC32C, 1, [Define to 1 to use software CRC-32C implementation (slicing-by-8).])
- PG_CRC32C_OBJS="pg_crc32c_sb8.o"
- AC_MSG_RESULT(slicing-by-8)
+ if test x"$USE_LOONGARCH_CRC32C" = x"1"; then
+ AC_DEFINE(USE_LOONGARCH_CRC32C, 1, [Define to 1 to use LoongArch CRCC instructions.])
+ PG_CRC32C_OBJS="pg_crc32c_loongarch.o"
+ AC_MSG_RESULT(LoongArch CRCC instructions)
+ else
+ AC_DEFINE(USE_SLICING_BY_8_CRC32C, 1, [Define to 1 to use software CRC-32C implementation (slicing-by-8).])
+ PG_CRC32C_OBJS="pg_crc32c_sb8.o"
+ AC_MSG_RESULT(slicing-by-8)
+ fi
fi
fi
fi
diff --git a/meson.build b/meson.build
index 0a11efc97a..2acb204003 100644
--- a/meson.build
+++ b/meson.build
@@ -2065,6 +2065,30 @@ int main(void)
cdata.set('USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK', 1)
have_optimized_crc = true
endif
+
+elif host_cpu == 'loongarch64'
+
+ prog = '''
+int main(void)
+{
+ unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+}
+'''
+
+ if cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and __builtin_loongarch_crcc_w_d_w',
+ args: test_c_args)
+ # Use LoongArch CRC instruction unconditionally
+ cdata.set('USE_LOONGARCH_CRC32C', 1)
+ have_optimized_crc = true
+ endif
+
endif
if not have_optimized_crc
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index ee209d6d70..d8a2985567 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -714,6 +714,9 @@
/* Define to 1 to build with LLVM based JIT support. (--with-llvm) */
#undef USE_LLVM
+/* Define to 1 to use LoongArch CRCC instructions. */
+#undef USE_LOONGARCH_CRC32C
+
/* Define to 1 to build with LZ4 support. (--with-lz4) */
#undef USE_LZ4
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 7f8779261c..d085f1dc00 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -58,6 +58,15 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+#elif defined(USE_LOONGARCH_CRC32C)
+/* Use LoongArch CRCC instructions. */
+
+#define COMP_CRC32C(crc, data, len) \
+ ((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
+#define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
+
+extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+
#elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
/*
diff --git a/src/port/meson.build b/src/port/meson.build
index 9d0cd93c43..deb354418d 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -92,6 +92,9 @@ replace_funcs_pos = [
['pg_crc32c_armv8_choose', 'USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK'],
['pg_crc32c_sb8', 'USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK'],
+ # loongarch
+ ['pg_crc32c_loongarch', 'USE_LOONGARCH_CRC32C'],
+
# generic fallback
['pg_crc32c_sb8', 'USE_SLICING_BY_8_CRC32C'],
]
diff --git a/src/port/pg_crc32c_loongarch.c b/src/port/pg_crc32c_loongarch.c
new file mode 100644
index 0000000000..2897920800
--- /dev/null
+++ b/src/port/pg_crc32c_loongarch.c
@@ -0,0 +1,73 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_crc32c_loongarch.c
+ * Compute CRC-32C checksum using LoongArch CRCC instructions
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_crc32c_loongarch.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "c.h"
+
+#include "port/pg_crc32c.h"
+
+pg_crc32c
+pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len)
+{
+ const unsigned char *p = data;
+ const unsigned char *pend = p + len;
+
+ /*
+ * Loongarch desktop and server chips support unaligned memory access by default.
+ * However, aligned memory access is significantly faster.
+ * Process leading bytes so that the loop below starts with a pointer aligned to eight bytes.
+ */
+ if (!PointerIsAligned(p, uint16) &&
+ p + 1 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
+ p += 1;
+ }
+ if (!PointerIsAligned(p, uint32) &&
+ p + 2 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
+ p += 2;
+ }
+ if (!PointerIsAligned(p, uint64) &&
+ p + 4 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
+ p += 4;
+ }
+
+ /* Process eight bytes at a time, as far as we can. */
+ while (p + 8 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_d_w(*(uint64 *) p, crc);
+ p += 8;
+ }
+
+ /* Process remaining 0-7 bytes. */
+ if (p + 4 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
+ p += 4;
+ }
+ if (p + 2 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
+ p += 2;
+ }
+ if (p < pend)
+ {
+ crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
+ }
+
+ return crc;
+}
--
2.41.0
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-07 11:01 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-08 03:06 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
@ 2023-08-08 06:38 ` John Naylor <[email protected]>
2023-08-08 07:22 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: John Naylor @ 2023-08-08 06:38 UTC (permalink / raw)
To: YANG Xudong <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]
On Tue, Aug 8, 2023 at 10:07 AM YANG Xudong <[email protected]> wrote:
> On 2023/8/7 19:01, John Naylor wrote:
> > The compilation test is found in c-compiler.m4, which still has all
> > logic for CFLAGS_CRC, including saving and restoring the old CFLAGS. Can
> > this also be simplified?
>
> Fixed the function in c-compiler.m4 by removing the function argument
> and the logic of handling CFLAGS and CFLAGS_CRC.
Looks good to me. It seems that platforms capable of running Postgres only
support 64 bit. If that ever changes, the compiler intrinsic test (with 8
byte CRC input) should still gate that well enough in autoconf, I believe,
so in v4 I added a comment to clarify this. The Meson build checks hostcpu
first for all platforms, and the patch is consistent with surrounding code.
In the attached 0002 addendum, I change a comment in configure.ac to
clarify "override" is referring to the runtime check for x86 and Arm, and
that LoongArch doesn't need one.
> > Can you confirm the alignment requirement -- it's not clear what the
> > intention is since "doesn't require" wasn't carried over. Is there any
> > documentation (or even a report in some other context) about aligned vs
> > unaligned memory access performance?
>
> It is in the official document that the alignment is not required.
>
>
https://github.com/loongson/la-softdev-convention/blob/master/la-softdev-convention.adoc#74-unaligne...
>
>
> However, I found this patch in LKML that shows great performance gain
> when using aligned memory access similar to this patch.
>
> https://lore.kernel.org/lkml/[email protected]/
>
> So I guess using aligned memory access is necessary and I have updated
> the comment in the code.
Okay, so it's not "necessary" in the sense that it's illegal, so I'm
thinking we can just re-use the Arm comment language, as in 0002.
v4 0001 is the same as v3, but with a draft commit message. I will squash
and commit this week, unless there is additional feedback.
--
John Naylor
EDB: http://www.enterprisedb.com
Attachments:
[text/x-patch] v4-0002-Some-minor-adjustemts-to-be-squashed.patch (5.0K, ../../CAFBsxsHqsNoORgdvrs1CsbdgeOTJ-tW21k=bPebZmjFsDwLkmg@mail.gmail.com/3-v4-0002-Some-minor-adjustemts-to-be-squashed.patch)
download | inline diff:
From 561893beb4e3e008196b3e571685503e25a243f1 Mon Sep 17 00:00:00 2001
From: John Naylor <[email protected]>
Date: Tue, 8 Aug 2023 12:58:07 +0700
Subject: [PATCH v4 2/2] Some minor adjustemts to be squashed
---
config/c-compiler.m4 | 4 ++++
configure | 7 +++++--
configure.ac | 7 +++++--
src/port/pg_crc32c_loongarch.c | 6 +++---
4 files changed, 17 insertions(+), 7 deletions(-)
diff --git a/config/c-compiler.m4 b/config/c-compiler.m4
index 7777ad6e90..bd3e6d6623 100644
--- a/config/c-compiler.m4
+++ b/config/c-compiler.m4
@@ -669,6 +669,10 @@ undefine([Ac_cachevar])dnl
# __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w
# intrinsic functions.
#
+# We test for the 8-byte variant since platforms capable of running
+# Postgres are 64-bit only (as of PG17), so we know CRC instructions
+# are available without a runtime check.
+#
# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics.
AC_DEFUN([PGAC_LOONGARCH_CRC32C_INTRINSICS],
[define([Ac_cachevar], [AS_TR_SH([pgac_cv_loongarch_crc32c_intrinsics])])dnl
diff --git a/configure b/configure
index fe0b02aa80..6a80e374f1 100755
--- a/configure
+++ b/configure
@@ -18119,8 +18119,11 @@ fi
# we're not targeting such a processor, but can nevertheless produce code that
# uses the CRC instructions, compile both, and select at runtime.
#
-# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
+# You can skip the runtime check by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
+#
+# If we are targeting a LoongArch processor, CRC instructions are
+# always available (at least on 64 bit), so no runtime check is needed.
if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
@@ -18139,8 +18142,8 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
+ # LoongArch CRCC instructions.
if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
- # LoongArch CRCC instructions.
USE_LOONGARCH_CRC32C=1
else
# fall back to slicing-by-8 algorithm, which doesn't require any
diff --git a/configure.ac b/configure.ac
index 57f0f836c7..6105af6996 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2130,8 +2130,11 @@ AC_SUBST(CFLAGS_CRC)
# we're not targeting such a processor, but can nevertheless produce code that
# uses the CRC instructions, compile both, and select at runtime.
#
-# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
+# You can skip the runtime check by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
+#
+# If we are targeting a LoongArch processor, CRC instructions are
+# always available (at least on 64 bit), so no runtime check is needed.
if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
@@ -2150,8 +2153,8 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
+ # LoongArch CRCC instructions.
if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
- # LoongArch CRCC instructions.
USE_LOONGARCH_CRC32C=1
else
# fall back to slicing-by-8 algorithm, which doesn't require any
diff --git a/src/port/pg_crc32c_loongarch.c b/src/port/pg_crc32c_loongarch.c
index 2897920800..db9da80e1b 100644
--- a/src/port/pg_crc32c_loongarch.c
+++ b/src/port/pg_crc32c_loongarch.c
@@ -23,9 +23,9 @@ pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len)
const unsigned char *pend = p + len;
/*
- * Loongarch desktop and server chips support unaligned memory access by default.
- * However, aligned memory access is significantly faster.
- * Process leading bytes so that the loop below starts with a pointer aligned to eight bytes.
+ * LoongArch doesn't require alignment, but aligned memory access is
+ * significantly faster. Process leading bytes so that the loop below
+ * starts with a pointer aligned to eight bytes.
*/
if (!PointerIsAligned(p, uint16) &&
p + 1 <= pend)
--
2.41.0
[text/x-patch] v4-0001-Use-native-CRC-instructions-on-LoongArch.patch (14.2K, ../../CAFBsxsHqsNoORgdvrs1CsbdgeOTJ-tW21k=bPebZmjFsDwLkmg@mail.gmail.com/4-v4-0001-Use-native-CRC-instructions-on-LoongArch.patch)
download | inline diff:
From 63666c12ee7533728b88a5028c28ea1e7c12a19a Mon Sep 17 00:00:00 2001
From: YANG Xudong <[email protected]>
Date: Tue, 8 Aug 2023 10:41:58 +0800
Subject: [PATCH v4 1/2] Use native CRC instructions on LoongArch
As with the Intel and Arm CRC instructions, compiler intrinsics
for them must be supported by the compiler. In contrast, no runtime
check is needed. Aligned memory access is faster, so use the coding
in pg_crc32c_armv8.c as the model.
YANG Xudong
Discussion: https://postgr.es/m/b522a0c5-e3b2-99cc-6387-58134fb88cbe%40ymatrix.cn
---
config/c-compiler.m4 | 29 ++++++++++++++
configure | 69 ++++++++++++++++++++++++++++----
configure.ac | 33 +++++++++++----
meson.build | 24 +++++++++++
src/include/pg_config.h.in | 3 ++
src/include/port/pg_crc32c.h | 9 +++++
src/port/meson.build | 3 ++
src/port/pg_crc32c_loongarch.c | 73 ++++++++++++++++++++++++++++++++++
8 files changed, 228 insertions(+), 15 deletions(-)
create mode 100644 src/port/pg_crc32c_loongarch.c
diff --git a/config/c-compiler.m4 b/config/c-compiler.m4
index 5be8f0f08d..7777ad6e90 100644
--- a/config/c-compiler.m4
+++ b/config/c-compiler.m4
@@ -661,3 +661,32 @@ if test x"$Ac_cachevar" = x"yes"; then
fi
undefine([Ac_cachevar])dnl
])# PGAC_ARMV8_CRC32C_INTRINSICS
+
+# PGAC_LOONGARCH_CRC32C_INTRINSICS
+# ---------------------------
+# Check if the compiler supports the LoongArch CRCC instructions, using
+# __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w,
+# __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w
+# intrinsic functions.
+#
+# If the intrinsics are supported, sets pgac_loongarch_crc32c_intrinsics.
+AC_DEFUN([PGAC_LOONGARCH_CRC32C_INTRINSICS],
+[define([Ac_cachevar], [AS_TR_SH([pgac_cv_loongarch_crc32c_intrinsics])])dnl
+AC_CACHE_CHECK(
+ [for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w],
+ [Ac_cachevar],
+[AC_LINK_IFELSE([AC_LANG_PROGRAM([],
+ [unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;])],
+ [Ac_cachevar=yes],
+ [Ac_cachevar=no])])
+if test x"$Ac_cachevar" = x"yes"; then
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+undefine([Ac_cachevar])dnl
+])# PGAC_LOONGARCH_CRC32C_INTRINSICS
diff --git a/configure b/configure
index 2e518c8007..fe0b02aa80 100755
--- a/configure
+++ b/configure
@@ -18062,6 +18062,47 @@ fi
fi
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w" >&5
+$as_echo_n "checking for __builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w and __builtin_loongarch_crcc_w_d_w... " >&6; }
+if ${pgac_cv_loongarch_crc32c_intrinsics+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+int
+main ()
+{
+unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ pgac_cv_loongarch_crc32c_intrinsics=yes
+else
+ pgac_cv_loongarch_crc32c_intrinsics=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_loongarch_crc32c_intrinsics" >&5
+$as_echo "$pgac_cv_loongarch_crc32c_intrinsics" >&6; }
+if test x"$pgac_cv_loongarch_crc32c_intrinsics" = x"yes"; then
+ pgac_loongarch_crc32c_intrinsics=yes
+fi
+
+
# Select CRC-32C implementation.
@@ -18080,7 +18121,7 @@ fi
#
# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
-if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x""; then
+if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
USE_SSE42_CRC32C=1
@@ -18098,10 +18139,15 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
- # fall back to slicing-by-8 algorithm, which doesn't require any
- # special CPU support.
- USE_SLICING_BY_8_CRC32C=1
- fi
+ if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
+ # LoongArch CRCC instructions.
+ USE_LOONGARCH_CRC32C=1
+ else
+ # fall back to slicing-by-8 algorithm, which doesn't require any
+ # special CPU support.
+ USE_SLICING_BY_8_CRC32C=1
+ fi
+ fi
fi
fi
fi
@@ -18142,12 +18188,21 @@ $as_echo "#define USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK 1" >>confdefs.h
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: ARMv8 CRC instructions with runtime check" >&5
$as_echo "ARMv8 CRC instructions with runtime check" >&6; }
else
+ if test x"$USE_LOONGARCH_CRC32C" = x"1"; then
+
+$as_echo "#define USE_LOONGARCH_CRC32C 1" >>confdefs.h
+
+ PG_CRC32C_OBJS="pg_crc32c_loongarch.o"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: LoongArch CRCC instructions" >&5
+$as_echo "LoongArch CRCC instructions" >&6; }
+ else
$as_echo "#define USE_SLICING_BY_8_CRC32C 1" >>confdefs.h
- PG_CRC32C_OBJS="pg_crc32c_sb8.o"
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: slicing-by-8" >&5
+ PG_CRC32C_OBJS="pg_crc32c_sb8.o"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: slicing-by-8" >&5
$as_echo "slicing-by-8" >&6; }
+ fi
fi
fi
fi
diff --git a/configure.ac b/configure.ac
index 3ebe1a796d..57f0f836c7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2108,6 +2108,12 @@ if test x"$pgac_armv8_crc32c_intrinsics" != x"yes"; then
PGAC_ARMV8_CRC32C_INTRINSICS([-march=armv8-a+crc])
fi
+# Check for LoongArch CRC intrinsics to do CRC calculations.
+#
+# Check if __builtin_loongarch_crcc_* intrinsics can be used
+# with the default compiler flags.
+PGAC_LOONGARCH_CRC32C_INTRINSICS()
+
AC_SUBST(CFLAGS_CRC)
# Select CRC-32C implementation.
@@ -2126,7 +2132,7 @@ AC_SUBST(CFLAGS_CRC)
#
# You can override this logic by setting the appropriate USE_*_CRC32 flag to 1
# in the template or configure command line.
-if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x""; then
+if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" && test x"$USE_SSE42_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_ARMV8_CRC32C" = x"" && test x"$USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK" = x"" && test x"$USE_LOONGARCH_CRC32C" = x""; then
# Use Intel SSE 4.2 if available.
if test x"$pgac_sse42_crc32_intrinsics" = x"yes" && test x"$SSE4_2_TARGETED" = x"1" ; then
USE_SSE42_CRC32C=1
@@ -2144,10 +2150,15 @@ if test x"$USE_SLICING_BY_8_CRC32C" = x"" && test x"$USE_SSE42_CRC32C" = x"" &&
if test x"$pgac_armv8_crc32c_intrinsics" = x"yes"; then
USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK=1
else
- # fall back to slicing-by-8 algorithm, which doesn't require any
- # special CPU support.
- USE_SLICING_BY_8_CRC32C=1
- fi
+ if test x"$pgac_loongarch_crc32c_intrinsics" = x"yes"; then
+ # LoongArch CRCC instructions.
+ USE_LOONGARCH_CRC32C=1
+ else
+ # fall back to slicing-by-8 algorithm, which doesn't require any
+ # special CPU support.
+ USE_SLICING_BY_8_CRC32C=1
+ fi
+ fi
fi
fi
fi
@@ -2175,9 +2186,15 @@ else
PG_CRC32C_OBJS="pg_crc32c_armv8.o pg_crc32c_sb8.o pg_crc32c_armv8_choose.o"
AC_MSG_RESULT(ARMv8 CRC instructions with runtime check)
else
- AC_DEFINE(USE_SLICING_BY_8_CRC32C, 1, [Define to 1 to use software CRC-32C implementation (slicing-by-8).])
- PG_CRC32C_OBJS="pg_crc32c_sb8.o"
- AC_MSG_RESULT(slicing-by-8)
+ if test x"$USE_LOONGARCH_CRC32C" = x"1"; then
+ AC_DEFINE(USE_LOONGARCH_CRC32C, 1, [Define to 1 to use LoongArch CRCC instructions.])
+ PG_CRC32C_OBJS="pg_crc32c_loongarch.o"
+ AC_MSG_RESULT(LoongArch CRCC instructions)
+ else
+ AC_DEFINE(USE_SLICING_BY_8_CRC32C, 1, [Define to 1 to use software CRC-32C implementation (slicing-by-8).])
+ PG_CRC32C_OBJS="pg_crc32c_sb8.o"
+ AC_MSG_RESULT(slicing-by-8)
+ fi
fi
fi
fi
diff --git a/meson.build b/meson.build
index 04ea348852..51582e143d 100644
--- a/meson.build
+++ b/meson.build
@@ -2073,6 +2073,30 @@ int main(void)
cdata.set('USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK', 1)
have_optimized_crc = true
endif
+
+elif host_cpu == 'loongarch64'
+
+ prog = '''
+int main(void)
+{
+ unsigned int crc = 0;
+ crc = __builtin_loongarch_crcc_w_b_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_h_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_w_w(0, crc);
+ crc = __builtin_loongarch_crcc_w_d_w(0, crc);
+
+ /* return computed value, to prevent the above being optimized away */
+ return crc == 0;
+}
+'''
+
+ if cc.links(prog, name: '__builtin_loongarch_crcc_w_b_w, __builtin_loongarch_crcc_w_h_w, __builtin_loongarch_crcc_w_w_w, and __builtin_loongarch_crcc_w_d_w',
+ args: test_c_args)
+ # Use LoongArch CRC instruction unconditionally
+ cdata.set('USE_LOONGARCH_CRC32C', 1)
+ have_optimized_crc = true
+ endif
+
endif
if not have_optimized_crc
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index ee209d6d70..d8a2985567 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -714,6 +714,9 @@
/* Define to 1 to build with LLVM based JIT support. (--with-llvm) */
#undef USE_LLVM
+/* Define to 1 to use LoongArch CRCC instructions. */
+#undef USE_LOONGARCH_CRC32C
+
/* Define to 1 to build with LZ4 support. (--with-lz4) */
#undef USE_LZ4
diff --git a/src/include/port/pg_crc32c.h b/src/include/port/pg_crc32c.h
index 7f8779261c..d085f1dc00 100644
--- a/src/include/port/pg_crc32c.h
+++ b/src/include/port/pg_crc32c.h
@@ -58,6 +58,15 @@ extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t le
extern pg_crc32c pg_comp_crc32c_armv8(pg_crc32c crc, const void *data, size_t len);
+#elif defined(USE_LOONGARCH_CRC32C)
+/* Use LoongArch CRCC instructions. */
+
+#define COMP_CRC32C(crc, data, len) \
+ ((crc) = pg_comp_crc32c_loongarch((crc), (data), (len)))
+#define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)
+
+extern pg_crc32c pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len);
+
#elif defined(USE_SSE42_CRC32C_WITH_RUNTIME_CHECK) || defined(USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK)
/*
diff --git a/src/port/meson.build b/src/port/meson.build
index 9d0cd93c43..deb354418d 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -92,6 +92,9 @@ replace_funcs_pos = [
['pg_crc32c_armv8_choose', 'USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK'],
['pg_crc32c_sb8', 'USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK'],
+ # loongarch
+ ['pg_crc32c_loongarch', 'USE_LOONGARCH_CRC32C'],
+
# generic fallback
['pg_crc32c_sb8', 'USE_SLICING_BY_8_CRC32C'],
]
diff --git a/src/port/pg_crc32c_loongarch.c b/src/port/pg_crc32c_loongarch.c
new file mode 100644
index 0000000000..2897920800
--- /dev/null
+++ b/src/port/pg_crc32c_loongarch.c
@@ -0,0 +1,73 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_crc32c_loongarch.c
+ * Compute CRC-32C checksum using LoongArch CRCC instructions
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/port/pg_crc32c_loongarch.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "c.h"
+
+#include "port/pg_crc32c.h"
+
+pg_crc32c
+pg_comp_crc32c_loongarch(pg_crc32c crc, const void *data, size_t len)
+{
+ const unsigned char *p = data;
+ const unsigned char *pend = p + len;
+
+ /*
+ * Loongarch desktop and server chips support unaligned memory access by default.
+ * However, aligned memory access is significantly faster.
+ * Process leading bytes so that the loop below starts with a pointer aligned to eight bytes.
+ */
+ if (!PointerIsAligned(p, uint16) &&
+ p + 1 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
+ p += 1;
+ }
+ if (!PointerIsAligned(p, uint32) &&
+ p + 2 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
+ p += 2;
+ }
+ if (!PointerIsAligned(p, uint64) &&
+ p + 4 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
+ p += 4;
+ }
+
+ /* Process eight bytes at a time, as far as we can. */
+ while (p + 8 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_d_w(*(uint64 *) p, crc);
+ p += 8;
+ }
+
+ /* Process remaining 0-7 bytes. */
+ if (p + 4 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_w_w(*(uint32 *) p, crc);
+ p += 4;
+ }
+ if (p + 2 <= pend)
+ {
+ crc = __builtin_loongarch_crcc_w_h_w(*(uint16 *) p, crc);
+ p += 2;
+ }
+ if (p < pend)
+ {
+ crc = __builtin_loongarch_crcc_w_b_w(*p, crc);
+ }
+
+ return crc;
+}
--
2.41.0
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-07 11:01 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-08 03:06 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-08 06:38 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
@ 2023-08-08 07:22 ` YANG Xudong <[email protected]>
2023-08-10 05:04 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: YANG Xudong @ 2023-08-08 07:22 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]
On 2023/8/8 14:38, John Naylor wrote:
>
> It seems that platforms capable of running Postgres
> only support 64 bit.
I think so.
> > So I guess using aligned memory access is necessary and I have updated
> > the comment in the code.
>
> Okay, so it's not "necessary" in the sense that it's illegal, so I'm
> thinking we can just re-use the Arm comment language, as in 0002.
Yes. I think it is similar to Arm.
> v4 0001 is the same as v3, but with a draft commit message. I will
> squash and commit this week, unless there is additional feedback.
Looks good to me. Thanks for the additional patch.
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-07 11:01 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-08 03:06 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-08 06:38 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-08 07:22 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
@ 2023-08-10 05:04 ` John Naylor <[email protected]>
2023-08-10 10:26 ` Re: [PATCH] Add loongarch native checksum implementation. Amit Kapila <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: John Naylor @ 2023-08-10 05:04 UTC (permalink / raw)
To: YANG Xudong <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]
On Tue, Aug 8, 2023 at 2:22 PM YANG Xudong <[email protected]> wrote:
>
> On 2023/8/8 14:38, John Naylor wrote:
>
> > v4 0001 is the same as v3, but with a draft commit message. I will
> > squash and commit this week, unless there is additional feedback.
>
> Looks good to me. Thanks for the additional patch.
I pushed this with another small comment change. Unfortunately, I didn't
glance at the buildfarm beforehand -- it seems many members are failing an
isolation check added by commit fa2e87494, including both loongarch64
members. I'll check back periodically.
https://buildfarm.postgresql.org/cgi-bin/show_status.pl
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-07 11:01 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-08 03:06 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-08 06:38 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-08 07:22 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-10 05:04 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
@ 2023-08-10 10:26 ` Amit Kapila <[email protected]>
2023-08-10 10:54 ` Re: [PATCH] Add loongarch native checksum implementation. Michael Paquier <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Amit Kapila @ 2023-08-10 10:26 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: YANG Xudong <[email protected]>; [email protected]; [email protected]; [email protected]
On Thu, Aug 10, 2023 at 10:35 AM John Naylor
<[email protected]> wrote:
>
> On Tue, Aug 8, 2023 at 2:22 PM YANG Xudong <[email protected]> wrote:
> >
> > On 2023/8/8 14:38, John Naylor wrote:
> >
> > > v4 0001 is the same as v3, but with a draft commit message. I will
> > > squash and commit this week, unless there is additional feedback.
> >
> > Looks good to me. Thanks for the additional patch.
>
> I pushed this with another small comment change. Unfortunately, I didn't glance at the buildfarm beforehand -- it seems many members are failing an isolation check added by commit fa2e87494, including both loongarch64 members. I'll check back periodically.
>
> https://buildfarm.postgresql.org/cgi-bin/show_status.pl
>
In MSVC build, on doing: perl mkvcbuild.pl after this commit, I am
facing the below error:
Generating configuration headers...
undefined symbol: USE_LOONGARCH_CRC32C at src/include/pg_config.h line
718 at ../postgresql/src/tools/msvc/Mkvcbuild.pm line 872.
Am I missing something or did the commit miss something?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-07 11:01 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-08 03:06 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-08 06:38 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-08 07:22 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-10 05:04 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-10 10:26 ` Re: [PATCH] Add loongarch native checksum implementation. Amit Kapila <[email protected]>
@ 2023-08-10 10:54 ` Michael Paquier <[email protected]>
2023-08-10 11:37 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: Michael Paquier @ 2023-08-10 10:54 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: John Naylor <[email protected]>; YANG Xudong <[email protected]>; [email protected]; [email protected]; [email protected]
On Thu, Aug 10, 2023 at 03:56:37PM +0530, Amit Kapila wrote:
> In MSVC build, on doing: perl mkvcbuild.pl after this commit, I am
> facing the below error:
> Generating configuration headers...
> undefined symbol: USE_LOONGARCH_CRC32C at src/include/pg_config.h line
> 718 at ../postgresql/src/tools/msvc/Mkvcbuild.pm line 872.
>
> Am I missing something or did the commit miss something?
Yes, the commit has missed the addition of USE_LOONGARCH_CRC32C in
Solution.pm. If you want to be consistent with pg_config.h.in, you
could add it just after USE_LLVM, for instance.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-07 11:01 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-08 03:06 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-08 06:38 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-08 07:22 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-10 05:04 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-10 10:26 ` Re: [PATCH] Add loongarch native checksum implementation. Amit Kapila <[email protected]>
2023-08-10 10:54 ` Re: [PATCH] Add loongarch native checksum implementation. Michael Paquier <[email protected]>
@ 2023-08-10 11:37 ` John Naylor <[email protected]>
2023-08-11 02:43 ` Re: [PATCH] Add loongarch native checksum implementation. Amit Kapila <[email protected]>
0 siblings, 1 reply; 31+ messages in thread
From: John Naylor @ 2023-08-10 11:37 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Amit Kapila <[email protected]>; YANG Xudong <[email protected]>; [email protected]; [email protected]; [email protected]
On Thu, Aug 10, 2023 at 5:54 PM Michael Paquier <[email protected]> wrote:
>
> On Thu, Aug 10, 2023 at 03:56:37PM +0530, Amit Kapila wrote:
> > In MSVC build, on doing: perl mkvcbuild.pl after this commit, I am
> > facing the below error:
> > Generating configuration headers...
> > undefined symbol: USE_LOONGARCH_CRC32C at src/include/pg_config.h line
> > 718 at ../postgresql/src/tools/msvc/Mkvcbuild.pm line 872.
> >
> > Am I missing something or did the commit miss something?
>
> Yes, the commit has missed the addition of USE_LOONGARCH_CRC32C in
> Solution.pm. If you want to be consistent with pg_config.h.in, you
> could add it just after USE_LLVM, for instance.
Oops, fixing now...
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 31+ messages in thread
* Re: [PATCH] Add loongarch native checksum implementation.
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-07 11:01 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-08 03:06 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-08 06:38 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-08 07:22 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-10 05:04 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-10 10:26 ` Re: [PATCH] Add loongarch native checksum implementation. Amit Kapila <[email protected]>
2023-08-10 10:54 ` Re: [PATCH] Add loongarch native checksum implementation. Michael Paquier <[email protected]>
2023-08-10 11:37 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
@ 2023-08-11 02:43 ` Amit Kapila <[email protected]>
0 siblings, 0 replies; 31+ messages in thread
From: Amit Kapila @ 2023-08-11 02:43 UTC (permalink / raw)
To: John Naylor <[email protected]>; +Cc: Michael Paquier <[email protected]>; YANG Xudong <[email protected]>; [email protected]; [email protected]; [email protected]
On Thu, Aug 10, 2023 at 5:07 PM John Naylor
<[email protected]> wrote:
>
> On Thu, Aug 10, 2023 at 5:54 PM Michael Paquier <[email protected]> wrote:
> >
> > On Thu, Aug 10, 2023 at 03:56:37PM +0530, Amit Kapila wrote:
> > > In MSVC build, on doing: perl mkvcbuild.pl after this commit, I am
> > > facing the below error:
> > > Generating configuration headers...
> > > undefined symbol: USE_LOONGARCH_CRC32C at src/include/pg_config.h line
> > > 718 at ../postgresql/src/tools/msvc/Mkvcbuild.pm line 872.
> > >
> > > Am I missing something or did the commit miss something?
> >
> > Yes, the commit has missed the addition of USE_LOONGARCH_CRC32C in
> > Solution.pm. If you want to be consistent with pg_config.h.in, you
> > could add it just after USE_LLVM, for instance.
>
> Oops, fixing now...
>
It is fixed now. Thanks!
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 31+ messages in thread
* [PATCH v4 1/1] Adjust extra lines generated by psql to be valid SQL comments.
@ 2023-07-26 21:35 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 31+ messages in thread
From: Nathan Bossart @ 2023-07-26 21:35 UTC (permalink / raw)
psql's --echo-hidden, --log-file, and --single-step options
generate extra lines to clearly separate queries from other output.
Presently, these extra lines are not valid SQL comments, which
makes them a hazard for anyone trying to copy/paste these decorated
queries into a client. This commit replaces the starting and
ending asterisks in these extra lines with forward slashes so that
they are valid SQL comments that can be copy/pasted without
incident.
Author: Kirk Wolak
Reviewed-by: Pavel Stehule, Laurenz Albe, Tom Lane, �lvaro Herrera, Andrey Borodin
Discussion: https://postgr.es/m/CACLU5mTFJRJYtbvmZ26txGgmXWQo0hkGhH2o3hEquUPmSbGtBw%40mail.gmail.com
---
src/bin/psql/command.c | 8 ++++----
src/bin/psql/common.c | 16 ++++++++--------
2 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 6733f008fd..1300869d79 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -5395,16 +5395,16 @@ echo_hidden_command(const char *query)
{
if (pset.echo_hidden != PSQL_ECHO_HIDDEN_OFF)
{
- printf(_("********* QUERY **********\n"
+ printf(_("/******** QUERY *********/\n"
"%s\n"
- "**************************\n\n"), query);
+ "/************************/\n\n"), query);
fflush(stdout);
if (pset.logfile)
{
fprintf(pset.logfile,
- _("********* QUERY **********\n"
+ _("/******** QUERY *********/\n"
"%s\n"
- "**************************\n\n"), query);
+ "/************************/\n\n"), query);
fflush(pset.logfile);
}
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index 5973df2e39..10ad1f2538 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -589,16 +589,16 @@ PSQLexec(const char *query)
if (pset.echo_hidden != PSQL_ECHO_HIDDEN_OFF)
{
- printf(_("********* QUERY **********\n"
+ printf(_("/******** QUERY *********/\n"
"%s\n"
- "**************************\n\n"), query);
+ "/************************/\n\n"), query);
fflush(stdout);
if (pset.logfile)
{
fprintf(pset.logfile,
- _("********* QUERY **********\n"
+ _("/******** QUERY *********/\n"
"%s\n"
- "**************************\n\n"), query);
+ "/************************/\n\n"), query);
fflush(pset.logfile);
}
@@ -1060,9 +1060,9 @@ SendQuery(const char *query)
char buf[3];
fflush(stderr);
- printf(_("***(Single step mode: verify command)*******************************************\n"
+ printf(_("/**(Single step mode: verify command)******************************************/\n"
"%s\n"
- "***(press return to proceed or enter x and return to cancel)********************\n"),
+ "/**(press return to proceed or enter x and return to cancel)*******************/\n"),
query);
fflush(stdout);
if (fgets(buf, sizeof(buf), stdin) != NULL)
@@ -1080,9 +1080,9 @@ SendQuery(const char *query)
if (pset.logfile)
{
fprintf(pset.logfile,
- _("********* QUERY **********\n"
+ _("/******** QUERY *********/\n"
"%s\n"
- "**************************\n\n"), query);
+ "/************************/\n\n"), query);
fflush(pset.logfile);
}
--
2.25.1
--UlVJffcvxoiEqYs2--
^ permalink raw reply [nested|flat] 31+ messages in thread
* [PATCH v4 1/1] Adjust extra lines generated by psql to be valid SQL comments.
@ 2023-07-26 21:35 Nathan Bossart <[email protected]>
0 siblings, 0 replies; 31+ messages in thread
From: Nathan Bossart @ 2023-07-26 21:35 UTC (permalink / raw)
psql's --echo-hidden, --log-file, and --single-step options
generate extra lines to clearly separate queries from other output.
Presently, these extra lines are not valid SQL comments, which
makes them a hazard for anyone trying to copy/paste these decorated
queries into a client. This commit replaces the starting and
ending asterisks in these extra lines with forward slashes so that
they are valid SQL comments that can be copy/pasted without
incident.
Author: Kirk Wolak
Reviewed-by: Pavel Stehule, Laurenz Albe, Tom Lane, �lvaro Herrera, Andrey Borodin
Discussion: https://postgr.es/m/CACLU5mTFJRJYtbvmZ26txGgmXWQo0hkGhH2o3hEquUPmSbGtBw%40mail.gmail.com
---
src/bin/psql/command.c | 8 ++++----
src/bin/psql/common.c | 16 ++++++++--------
2 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 6733f008fd..1300869d79 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -5395,16 +5395,16 @@ echo_hidden_command(const char *query)
{
if (pset.echo_hidden != PSQL_ECHO_HIDDEN_OFF)
{
- printf(_("********* QUERY **********\n"
+ printf(_("/******** QUERY *********/\n"
"%s\n"
- "**************************\n\n"), query);
+ "/************************/\n\n"), query);
fflush(stdout);
if (pset.logfile)
{
fprintf(pset.logfile,
- _("********* QUERY **********\n"
+ _("/******** QUERY *********/\n"
"%s\n"
- "**************************\n\n"), query);
+ "/************************/\n\n"), query);
fflush(pset.logfile);
}
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index 5973df2e39..10ad1f2538 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -589,16 +589,16 @@ PSQLexec(const char *query)
if (pset.echo_hidden != PSQL_ECHO_HIDDEN_OFF)
{
- printf(_("********* QUERY **********\n"
+ printf(_("/******** QUERY *********/\n"
"%s\n"
- "**************************\n\n"), query);
+ "/************************/\n\n"), query);
fflush(stdout);
if (pset.logfile)
{
fprintf(pset.logfile,
- _("********* QUERY **********\n"
+ _("/******** QUERY *********/\n"
"%s\n"
- "**************************\n\n"), query);
+ "/************************/\n\n"), query);
fflush(pset.logfile);
}
@@ -1060,9 +1060,9 @@ SendQuery(const char *query)
char buf[3];
fflush(stderr);
- printf(_("***(Single step mode: verify command)*******************************************\n"
+ printf(_("/**(Single step mode: verify command)******************************************/\n"
"%s\n"
- "***(press return to proceed or enter x and return to cancel)********************\n"),
+ "/**(press return to proceed or enter x and return to cancel)*******************/\n"),
query);
fflush(stdout);
if (fgets(buf, sizeof(buf), stdin) != NULL)
@@ -1080,9 +1080,9 @@ SendQuery(const char *query)
if (pset.logfile)
{
fprintf(pset.logfile,
- _("********* QUERY **********\n"
+ _("/******** QUERY *********/\n"
"%s\n"
- "**************************\n\n"), query);
+ "/************************/\n\n"), query);
fflush(pset.logfile);
}
--
2.25.1
--UlVJffcvxoiEqYs2--
^ permalink raw reply [nested|flat] 31+ messages in thread
end of thread, other threads:[~2023-08-11 02:43 UTC | newest]
Thread overview: 31+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-10-11 01:03 [PATCH 2/4] Write WAL for empty nbtree index build Kyotaro Horiguchi <[email protected]>
2018-10-11 01:03 [PATCH 2/4] Write WAL for empty nbtree index build Kyotaro Horiguchi <[email protected]>
2018-10-11 01:03 [PATCH 2/4] Write WAL for empty nbtree index build Kyotaro Horiguchi <[email protected]>
2018-10-11 01:03 [PATCH 3/3] Write WAL for empty nbtree index build Kyotaro Horiguchi <[email protected]>
2018-10-11 01:03 [PATCH 2/4] Write WAL for empty nbtree index build Kyotaro Horiguchi <[email protected]>
2019-11-07 06:33 [PATCH 5/6] Preserve index dependencies on collation during pg_upgrade Julien Rouhaud <[email protected]>
2023-06-13 10:26 Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-14 02:20 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-06-15 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-06-16 01:28 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 02:15 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-05 07:11 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-07-26 03:16 ` Re: [PATCH] Add loongarch native checksum implementation. Michael Paquier <[email protected]>
2023-07-26 04:37 ` Re: [PATCH] Add loongarch native checksum implementation. Nathan Bossart <[email protected]>
2023-07-26 05:36 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-26 07:38 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-07-06 07:14 ` Re: Re: [PATCH] Add loongarch native checksum implementation. huchangqi <[email protected]>
2023-07-06 09:48 ` Re: [PATCH] Add loongarch native checksum implementation. Daniel Gustafsson <[email protected]>
2023-07-06 10:30 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-07-26 01:25 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-07 11:01 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-08 03:06 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-08 06:38 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-08 07:22 ` Re: [PATCH] Add loongarch native checksum implementation. YANG Xudong <[email protected]>
2023-08-10 05:04 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-10 10:26 ` Re: [PATCH] Add loongarch native checksum implementation. Amit Kapila <[email protected]>
2023-08-10 10:54 ` Re: [PATCH] Add loongarch native checksum implementation. Michael Paquier <[email protected]>
2023-08-10 11:37 ` Re: [PATCH] Add loongarch native checksum implementation. John Naylor <[email protected]>
2023-08-11 02:43 ` Re: [PATCH] Add loongarch native checksum implementation. Amit Kapila <[email protected]>
2023-07-26 21:35 [PATCH v4 1/1] Adjust extra lines generated by psql to be valid SQL comments. Nathan Bossart <[email protected]>
2023-07-26 21:35 [PATCH v4 1/1] Adjust extra lines generated by psql to be valid SQL comments. Nathan Bossart <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox