public inbox for [email protected]
help / color / mirror / Atom feedFrom: Nadav Shatz <[email protected]>
To: Tatsuo Ishii <[email protected]>
Cc: [email protected]
Subject: Re: Proposal: Recent mutated table tracking in memory
Date: Wed, 15 Apr 2026 15:17:17 +0300
Message-ID: <CACeKOO3k8K0=u9AKmctyFVivrCuMtN5OdcCDXt0ew=qwbrQGcA@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
<[email protected]>
<CACeKOO2VQ1o9TGk77-hfJzwcGeEd0cCx7AqEDXphqkG4xzG+7w@mail.gmail.com>
<[email protected]>
Hi Tatsuo,
hank you for the detailed review. Attached patch addresses all items.
memqcache bug fix
-----------------
Good catch. The root cause: pool_set_writing_transaction() was
explicitly skipping dml_adaptive_global, so
pool_is_writing_transaction() always returned false in this mode.
The query cache fetch guard at pool_proto_modules.c:270
(!pool_is_writing_transaction()) then served stale cached results
after DML in the same transaction.
Fix: pool_set_writing_transaction() now sets the flag for
dml_adaptive_global (only 'off' and 'dml_adaptive' skip it). This
ensures the query cache is properly bypassed after writes within
the same transaction.
Removed dead query parse cache code (~700 lines)
-------------------------------------------------
You're right -- pool_track_table_mutation_get_cached_parse,
pool_track_table_mutation_cache_parse, and
pool_track_table_mutation_normalize_and_hash were never called.
These were leftover from an earlier design where we planned to
cache SQL parse results in shared memory. The feature ended up
using pgpool's existing parser directly, and this code was never
wired up.
Removed: QueryParseCache and QueryParseEntry structs, all related
static functions, the TRACK_TABLE_MUTATION_QUERY_SEM semaphore,
and the track_table_mutation_query_buckets /
track_table_mutation_query_parse_cache_size configuration
parameters. This also reduces shared memory usage from ~6.4 MB
to ~80 KB with default settings.
check_object_relationship_list scope
-------------------------------------
You're correct -- dml_adaptive_global does not use
dml_adaptive_object_relationship_list. Changed
check_object_relationship_list() to check for DLBOW_DML_ADAPTIVE
only, not DLBOW_IS_DML_ADAPTIVE (which includes global).
Documentation fixes
-------------------
- Removed "(Lagless Replica Reads)" from section title and
"lagless" language from description.
- Described fallback behavior when neither
replication_delay_source_cmd nor delay_threshold_by_time is
configured (TTL stays at 100ms default minimum).
- "query cache" references removed (the query parse cache is gone).
- Added 128-table-per-SELECT limit to Limitations section
(uses POOL_MAX_SELECT_OIDS).
Code style fixes
----------------
- DLBOW_IS_DML_ADAPTIVE() calls no longer split across lines.
- Split the long errmsg line in
is_select_object_in_temp_write_list.
- Removed redundant is_adaptive variable in
is_select_object_in_temp_write_list (the check at function
entry already guarantees it).
Thanks!
On Wed, Apr 15, 2026 at 1:43 AM Tatsuo Ishii <[email protected]> wrote:
> Hi Nadav,
>
> > Hi Tatsuo,
> >
> > Looks good to me thanks!
> >
> > Please go ahead with your review. waiting to hear back from you.
>
> Here are the code review results.
>
> diff --git a/doc/src/sgml/loadbalance.sgml b/doc/src/sgml/loadbalance.sgml
> index 9e1e7b39b..7384ce81a 100644
> --- a/doc/src/sgml/loadbalance.sgml
> +++ b/doc/src/sgml/loadbalance.sgml
> :
> + <sect2 id="runtime-config-table-mutation-map">
> + <title>Table Mutation Map Configuration (Lagless Replica Reads)</title>
>
> "(Lagless Replica Reads)" sounds like an advertisement to me. It
> should be removed.
>
> + <para>
> + These parameters configure the track table mutation feature, which is
> activated by setting
> + <xref linkend="guc-disable-load-balance-on-write"> to
> <literal>dml_adaptive_global</literal>.
> + The feature tracks recently written tables to prevent stale reads from
> replica nodes during
> + replication lag, implementing the "lagless" architecture pattern for
> distributed systems
> + with read replicas.
>
> I think the feature does not guarantee "lagless" anytime, in all cases.
>
> + <para>
> + This feature requires time-based replication delay monitoring. This
> can be provided by either
> + <xref linkend="guc-replication-delay-source-cmd"> (external command
> mode) or by setting
> + <xref linkend="guc-delay-threshold-by-time"> (which uses
> <literal>pg_stat_replication.replay_lag</literal>
> + from PostgreSQL 10+). At least one of these must be configured for the
> TTL calculation to work.
>
> If one of these is not set, what happens? Error? Need to describe it.
>
> + </para>
> +
> + <warning>
> + <para>
> + Enabling <literal>dml_adaptive_global</literal> increases shared
> memory consumption. With default settings,
> + the feature requires approximately 6.4 MB of shared memory (0.1 MB
> for table tracking + 6.3 MB for query cache).
>
> "query cache" should be "query parse cache".
>
> + Memory usage scales with configuration parameters:
> + </para>
> + <itemizedlist>
> + <listitem>
> + <para>
> + Table tracking: <literal>track_table_mutation_table_size * 40
> bytes</literal> (default: 2048 * 40 = ~80 KB)
> + </para>
> + </listitem>
> + <listitem>
> + <para>
> + Query cache: <literal>track_table_mutation_query_parse_cache_size *
> 640 bytes</literal> (default: 10000 * 640 = ~6.3 MB)
>
> "query cache" should be "query parse cache".
>
> + <title>Limitations</title>
>
> I think number of tables tacked in a SELECT is limited to 8. It should
> be mentioned.
>
> diff --git a/src/context/pool_query_context.c
> b/src/context/pool_query_context.c
> index a056ac596..0190d3673 100644
> --- a/src/context/pool_query_context.c
> +++ b/src/context/pool_query_context.c
> @@ -1828,15 +1829,23 @@ is_in_list(char *name, List *list)
> static bool
> is_select_object_in_temp_write_list(Node *node, void *context)
> {
> - if (node == NULL || pool_config->disable_load_balance_on_write !=
> DLBOW_DML_ADAPTIVE)
> + if (node == NULL ||
> + !DLBOW_IS_DML_ADAPTIVE(
> +
> pool_config->disable_load_balance_on_write))
>
> You don't need to split the line.
>
> + is_adaptive = DLBOW_IS_DML_ADAPTIVE(
> +
> pool_config->disable_load_balance_on_write);
>
> You don't need to split the line.
>
> - if (pool_config->disable_load_balance_on_write ==
> DLBOW_DML_ADAPTIVE && session_context->is_in_transaction)
> + if (is_adaptive &&
> + session_context->is_in_transaction)
> {
> ereport(DEBUG1,
>
> (errmsg("is_select_object_in_temp_write_list: \"%s\", found relation
> \"%s\"", (char *) context, rgv->relname)));
> This line is too long. Please split.
>
> @@ -1880,7 +1889,13 @@ static char
> *get_associated_object_from_dml_adaptive_relations
> void
> check_object_relationship_list(char *name, bool is_func_name)
> {
> - if (pool_config->disable_load_balance_on_write ==
> DLBOW_DML_ADAPTIVE &&
> pool_config->parsed_dml_adaptive_object_relationship_list)
> + bool is_adaptive;
> +
> + is_adaptive = DLBOW_IS_DML_ADAPTIVE(
> +
> pool_config->disable_load_balance_on_write);
>
> I wrote in the commit message:
>
> modifications are only detected in the same transaction). Note,
> however, you cannot use dml_adaptive_object_relationship_list to track
> dependency among table and other objects.
>
> In my understanding the feature does not use
> dml_adaptive_object_relationship_list. If this is correct, why
> check_object_relationship_list() is called here in case
> dml_adaptive_global? If the feature uses
> dml_adaptive_object_relationship_list, test cases should be included.
>
> diff --git a/src/utils/pool_track_table_mutation.c
> b/src/utils/pool_track_table_mutation.c
> new file mode 100644
> index 000000000..9be46b28f
> --- /dev/null
> +++ b/src/utils/pool_track_table_mutation.c
>
> It seems following functions are not used anywhere. I wonder if this
> feature actually use "query parse cache".
>
> pool_track_table_mutation_get_cached_parse
> pool_track_table_mutation_cache_parse
> pool_track_table_mutation_normalize_and_hash
>
> Besides the code review, I mutated one of regression tests to check
> whether the feature co exists with in the existing memory query cache
> feature. After attached patch applied, I ran 006.memqcache and got the
> following result.
>
> cd src/test/regression
> ./regress.sh 006
> creating pgpool-II temporary installation ...
> moving pgpool_setup to temporary installation path ...
> moving watchdog_setup to temporary installation path ...
> using pgpool-II at
> /home/t-ishii/work/Pgpool-II/current/pgpool2/src/test/regression/temp/installed
> *************************
> REGRESSION MODE : install
> Pgpool-II version : pgpool-II version 4.8devel (mitsukakeboshi)
> Pgpool-II install path :
> /home/t-ishii/work/Pgpool-II/current/pgpool2/src/test/regression/temp/installed
> PostgreSQL bin : /usr/local/pgsql/bin
> PostgreSQL Major version : 18
> pgbench : /usr/local/pgsql/bin/pgbench
> PostgreSQL jdbc :
> /usr/local/pgsql/share/postgresql-9.2-1003.jdbc4.jar
> *************************
> testing 006.memqcache...failed.
> out of 1 ok:0 failed:1 timeout:0
>
> log/006.memqcache shows:
>
> ../expected.txt result.txt differ: char 1, line 1
>
> So I checked the test script and found the error was generated by a
> Java program test.
>
> java jdbctest > result.txt 2>&1
> cmp ../expected.txt result.txt
> if [ $? != 0 ];then
> ./shutdownall
> exit 1
> fi
>
> In jdbctest.java:
>
> /*
> * Cache test in an explicit transaction
> */
> conn.setAutoCommit(false);
> // execute DML. This should prevent SELECTs from using
> query cache in the transaction.
> sql = "UPDATE t1 SET i = 2;";
> pst = conn.createStatement();
> pst.executeUpdate(sql);
> pst.close();
> // should not use the cache and should return "2", rather
> than "1"
> prest = conn.prepareStatement("SELECT * FROM t1");
> rs = prest.executeQuery();
>
> The expected file (expected.txt) has "2" but the result file
> (testdir/result.txt) was "1". This is the reason why the test
> failed. I wonder if there's something wrong with the feature when the
> query cache is enabled. Can you look into this?
>
> Regards,
> --
> Tatsuo Ishii
> SRA OSS K.K.
> English: http://www.sraoss.co.jp/index_en/
> Japanese:http://www.sraoss.co.jp
>
--
Nadav Shatz
Tailor Brands | CTO
Attachments:
[application/octet-stream] v2-0001-address-review.patch (34.2K, 3-v2-0001-address-review.patch)
download | inline diff:
From ceebe131825941e1d49dd071bf32ffcb021339a5 Mon Sep 17 00:00:00 2001
From: Nadav Shatz <[email protected]>
Date: Wed, 15 Apr 2026 11:44:21 +0300
Subject: [PATCH] Address review: remove query parse cache, fix memqcache bug.
- Remove dead query parse cache code (QueryParseCache,
QueryParseEntry, and all related functions). These were
never wired up; the feature uses pgpool's existing parser.
This removes ~700 lines, the TRACK_TABLE_MUTATION_QUERY_SEM
semaphore, and the track_table_mutation_query_buckets and
track_table_mutation_query_parse_cache_size parameters.
- Fix stale read from query cache (memqcache) when
dml_adaptive_global is active. pool_set_writing_transaction()
was skipping dml_adaptive_global, so pool_is_writing_transaction()
always returned false, allowing cached results after DML in the
same transaction. Now dml_adaptive_global sets the flag so the
query cache is properly skipped after writes.
- Restrict check_object_relationship_list() to dml_adaptive only.
dml_adaptive_global does not use
dml_adaptive_object_relationship_list.
- Fix docs: remove marketing language, describe behavior when
no delay source is configured, add 128-table-per-SELECT limit
to limitations, fix line length and split issues.
Author: Nadav Shatz <[email protected]>
---
doc/src/sgml/loadbalance.sgml | 82 +--
src/config/pool_config_variables.c | 24 -
src/context/pool_query_context.c | 31 +-
src/context/pool_session_context.c | 10 +-
src/include/pool.h | 3 +-
src/include/pool_config.h | 4 -
src/include/utils/pool_track_table_mutation.h | 80 ---
src/sample/pgpool.conf.sample-stream | 13 +-
src/tools/pgindent/typedefs.list | 2 -
src/utils/pool_track_table_mutation.c | 550 +-----------------
10 files changed, 42 insertions(+), 757 deletions(-)
diff --git a/doc/src/sgml/loadbalance.sgml b/doc/src/sgml/loadbalance.sgml
index 7384ce81a..d4fbcf1a5 100644
--- a/doc/src/sgml/loadbalance.sgml
+++ b/doc/src/sgml/loadbalance.sgml
@@ -1209,14 +1209,13 @@ dml_adaptive_object_relationship_list = 'table_1:table_2'
</sect2>
<sect2 id="runtime-config-table-mutation-map">
- <title>Table Mutation Map Configuration (Lagless Replica Reads)</title>
+ <title>Table Mutation Tracking Configuration</title>
<para>
These parameters configure the track table mutation feature, which is activated by setting
<xref linkend="guc-disable-load-balance-on-write"> to <literal>dml_adaptive_global</literal>.
The feature tracks recently written tables to prevent stale reads from replica nodes during
- replication lag, implementing the "lagless" architecture pattern for distributed systems
- with read replicas.
+ replication lag.
</para>
<para>
@@ -1229,30 +1228,16 @@ dml_adaptive_object_relationship_list = 'table_1:table_2'
This feature requires time-based replication delay monitoring. This can be provided by either
<xref linkend="guc-replication-delay-source-cmd"> (external command mode) or by setting
<xref linkend="guc-delay-threshold-by-time"> (which uses <literal>pg_stat_replication.replay_lag</literal>
- from PostgreSQL 10+). At least one of these must be configured for the TTL calculation to work.
+ from PostgreSQL 10+). If neither is configured, the TTL remains at its default minimum value
+ (100 milliseconds) and is never updated based on actual replication delay, which may result
+ in suboptimal routing decisions.
</para>
<warning>
<para>
Enabling <literal>dml_adaptive_global</literal> increases shared memory consumption. With default settings,
- the feature requires approximately 6.4 MB of shared memory (0.1 MB for table tracking + 6.3 MB for query cache).
- Memory usage scales with configuration parameters:
- </para>
- <itemizedlist>
- <listitem>
- <para>
- Table tracking: <literal>track_table_mutation_table_size * 40 bytes</literal> (default: 2048 * 40 = ~80 KB)
- </para>
- </listitem>
- <listitem>
- <para>
- Query cache: <literal>track_table_mutation_query_parse_cache_size * 640 bytes</literal> (default: 10000 * 640 = ~6.3 MB)
- </para>
- </listitem>
- </itemizedlist>
- <para>
- For high-traffic systems with large cache sizes (e.g., <literal>track_table_mutation_query_parse_cache_size = 100000</literal>),
- memory usage can reach 64 MB or more. Consider your system's available shared memory when using <literal>dml_adaptive_global</literal>.
+ the feature requires approximately 80 KB of shared memory for table tracking:
+ <literal>track_table_mutation_table_size * 40 bytes</literal> (default: 2048 * 40 = ~80 KB).
</para>
</warning>
@@ -1364,43 +1349,6 @@ dml_adaptive_object_relationship_list = 'table_1:table_2'
</listitem>
</varlistentry>
- <varlistentry id="guc-track-table-mutation-query-buckets" xreflabel="track_table_mutation_query_buckets">
- <term><varname>track_table_mutation_query_buckets</varname> (<type>integer</type>)
- <indexterm>
- <primary><varname>track_table_mutation_query_buckets</varname> configuration parameter</primary>
- </indexterm>
- </term>
- <listitem>
- <para>
- Number of hash buckets for the query parse cache. The cache stores normalized
- query strings mapped to their table dependencies to avoid repeated parsing.
- </para>
- <para>
- Valid range: 64-65536. Default is <literal>2048</literal>.
- This parameter can only be set at server start.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry id="guc-track-table-mutation-query-parse-cache-size" xreflabel="track_table_mutation_query_parse_cache_size">
- <term><varname>track_table_mutation_query_parse_cache_size</varname> (<type>integer</type>)
- <indexterm>
- <primary><varname>track_table_mutation_query_parse_cache_size</varname> configuration parameter</primary>
- </indexterm>
- </term>
- <listitem>
- <para>
- Maximum number of query parse results to cache. Uses LRU eviction when full.
- Larger caches reduce parsing overhead but consume more shared memory.
- </para>
- <para>
- Valid range: 100-1000000. Default is <literal>10000</literal>.
- Memory usage: approximately 640 bytes per entry (~6.3 MB for default, ~64 MB for 100000 entries).
- This parameter can only be set at server start.
- </para>
- </listitem>
- </varlistentry>
-
</variablelist>
<sect3 id="runtime-config-track-table-mutation-example">
@@ -1422,20 +1370,19 @@ replication_delay_source_timeout = 10
# Option B: Use pg_stat_replication replay_lag (PG 10+)
# delay_threshold_by_time = 1000
-# Adjust cache sizes based on workload (increases memory usage)
+# Adjust table map size based on workload
track_table_mutation_table_size = 4096
-track_table_mutation_query_parse_cache_size = 50000
</programlisting>
<para>
- Total shared memory required for above configuration: approximately 31.2 MB (31 MB query cache + 0.2 MB table map + overhead).
- Default configuration (10000 query cache entries, 2048 tables) requires approximately 6.4 MB.
+ Shared memory required for above configuration: approximately 160 KB for the table map.
+ Default configuration (2048 tables) requires approximately 80 KB.
</para>
</sect3>
<sect3 id="runtime-config-track-table-mutation-limitations">
<title>Limitations</title>
<para>
- The track table mutation feature has the following limitation:
+ The track table mutation feature has the following limitations:
</para>
<itemizedlist>
<listitem>
@@ -1444,6 +1391,13 @@ track_table_mutation_query_parse_cache_size = 50000
containing data modification is executed, the table mutation is not recorded.
</para>
</listitem>
+ <listitem>
+ <para>
+ A maximum of 128 tables can be tracked per SELECT query for staleness checking.
+ This limit is shared with the query cache subsystem
+ (<literal>POOL_MAX_SELECT_OIDS</literal>).
+ </para>
+ </listitem>
</itemizedlist>
<para>
If your application uses prepared statements and requires read-after-write consistency,
diff --git a/src/config/pool_config_variables.c b/src/config/pool_config_variables.c
index d5f4fb605..bbd65b176 100644
--- a/src/config/pool_config_variables.c
+++ b/src/config/pool_config_variables.c
@@ -2462,30 +2462,6 @@ static struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
- {
- {"track_table_mutation_query_buckets",
- CFGCXT_INIT, LOAD_BALANCE_CONFIG,
- "Number of hash buckets for query parse cache.",
- CONFIG_VAR_TYPE_INT, false, 0
- },
- &g_pool_config.track_table_mutation_query_buckets,
- 2048,
- 64, 65536,
- NULL, NULL, NULL
- },
-
- {
- {"track_table_mutation_query_parse_cache_size",
- CFGCXT_INIT, LOAD_BALANCE_CONFIG,
- "Maximum number of entries in query parse cache.",
- CONFIG_VAR_TYPE_INT, false, 0
- },
- &g_pool_config.track_table_mutation_query_parse_cache_size,
- 10000,
- 100, 1000000,
- NULL, NULL, NULL
- },
-
/* End-of-list marker */
EMPTY_CONFIG_INT
};
diff --git a/src/context/pool_query_context.c b/src/context/pool_query_context.c
index 0190d3673..c20a3a420 100644
--- a/src/context/pool_query_context.c
+++ b/src/context/pool_query_context.c
@@ -1830,27 +1830,25 @@ static bool
is_select_object_in_temp_write_list(Node *node, void *context)
{
if (node == NULL ||
- !DLBOW_IS_DML_ADAPTIVE(
- pool_config->disable_load_balance_on_write))
+ !DLBOW_IS_DML_ADAPTIVE(pool_config->disable_load_balance_on_write))
return false;
if (IsA(node, RangeVar))
{
RangeVar *rgv = (RangeVar *) node;
POOL_SESSION_CONTEXT *session_context;
- bool is_adaptive;
session_context = pool_get_session_context(false);
- is_adaptive = DLBOW_IS_DML_ADAPTIVE(
- pool_config->disable_load_balance_on_write);
- if (is_adaptive &&
- session_context->is_in_transaction)
+ if (session_context->is_in_transaction)
{
ereport(DEBUG1,
- (errmsg("is_select_object_in_temp_write_list: \"%s\", found relation \"%s\"", (char *) context, rgv->relname)));
+ (errmsg("is_select_object_in_temp_write_list:"
+ " \"%s\", found relation \"%s\"",
+ (char *) context, rgv->relname)));
- return is_in_list(rgv->relname, session_context->transaction_temp_write_list);
+ return is_in_list(rgv->relname,
+ session_context->transaction_temp_write_list);
}
}
@@ -1891,8 +1889,9 @@ check_object_relationship_list(char *name, bool is_func_name)
{
bool is_adaptive;
- is_adaptive = DLBOW_IS_DML_ADAPTIVE(
- pool_config->disable_load_balance_on_write);
+ is_adaptive =
+ (pool_config->disable_load_balance_on_write ==
+ DLBOW_DML_ADAPTIVE);
if (is_adaptive &&
pool_config->parsed_dml_adaptive_object_relationship_list)
@@ -1902,8 +1901,8 @@ check_object_relationship_list(char *name, bool is_func_name)
if (session_context->is_in_transaction)
{
char *right_token =
- get_associated_object_from_dml_adaptive_relations
- (name, is_func_name ? OBJECT_TYPE_FUNCTION : OBJECT_TYPE_RELATION);
+ get_associated_object_from_dml_adaptive_relations
+ (name, is_func_name ? OBJECT_TYPE_FUNCTION : OBJECT_TYPE_RELATION);
if (right_token)
{
@@ -1989,9 +1988,9 @@ dml_adaptive(Node *node, char *query)
* transactions.
*/
int dlbow =
- pool_config->disable_load_balance_on_write;
+ pool_config->disable_load_balance_on_write;
List *wlist =
- session_context->transaction_temp_write_list;
+ session_context->transaction_temp_write_list;
if (dlbow == DLBOW_DML_ADAPTIVE_GLOBAL &&
is_commit_query(node) &&
@@ -2231,7 +2230,7 @@ where_to_send_main_replica(POOL_QUERY_CONTEXT *query_context, char *query, Node
bool force_primary = false;
int lb_node;
POOL_QUERY_CONTEXT *qctx =
- session_context->query_context;
+ session_context->query_context;
if (pool_track_table_mutation_in_cold_start())
{
diff --git a/src/context/pool_session_context.c b/src/context/pool_session_context.c
index 05d0b635b..be30f1a7c 100644
--- a/src/context/pool_session_context.c
+++ b/src/context/pool_session_context.c
@@ -740,13 +740,15 @@ void
pool_set_writing_transaction(void)
{
/*
- * If disable_load_balance_on_write is 'off' or 'dml_adaptive' or
- * 'dml_adaptive_global', then never turn on writing transaction flag.
+ * If disable_load_balance_on_write is 'off' or 'dml_adaptive', then never
+ * turn on writing transaction flag. For dml_adaptive_global we do set it
+ * so that the query cache (memqcache) is properly skipped after DML
+ * within the same transaction.
*/
if (pool_config->disable_load_balance_on_write !=
DLBOW_OFF &&
- !DLBOW_IS_DML_ADAPTIVE(
- pool_config->disable_load_balance_on_write))
+ pool_config->disable_load_balance_on_write !=
+ DLBOW_DML_ADAPTIVE)
{
pool_get_session_context(false)->writing_transaction = true;
ereport(DEBUG5,
diff --git a/src/include/pool.h b/src/include/pool.h
index 0e901691a..79d7988fc 100644
--- a/src/include/pool.h
+++ b/src/include/pool.h
@@ -424,7 +424,7 @@ typedef enum
#define Min(x, y) ((x) < (y) ? (x) : (y))
-#define MAX_NUM_SEMAPHORES 10
+#define MAX_NUM_SEMAPHORES 9
#define CONN_COUNTER_SEM 0
#define REQUEST_INFO_SEM 1
#define QUERY_CACHE_STATS_SEM 2
@@ -435,7 +435,6 @@ typedef enum
#define MAIN_EXIT_HANDLER_SEM 7 /* used in exit_hander in pgpool main
* process */
#define TRACK_TABLE_MUTATION_TABLE_SEM 8
-#define TRACK_TABLE_MUTATION_QUERY_SEM 9
#define MAX_REQUEST_QUEUE_SIZE 10
#define MAX_SEC_WAIT_FOR_CLUSTER_TRANSACTION 10 /* time in seconds to keep
diff --git a/src/include/pool_config.h b/src/include/pool_config.h
index ae507dc60..b8abadd50 100644
--- a/src/include/pool_config.h
+++ b/src/include/pool_config.h
@@ -382,10 +382,6 @@ typedef struct
int track_table_mutation_table_buckets; /* hash buckets for table
* map */
int track_table_mutation_table_size; /* max table map entries */
- int track_table_mutation_query_buckets; /* hash buckets for query
- * cache */
- int track_table_mutation_query_parse_cache_size; /* max query cache
- * entries */
char *failover_command; /* execute command when failover happens */
char *follow_primary_command; /* execute command when failover is
diff --git a/src/include/utils/pool_track_table_mutation.h b/src/include/utils/pool_track_table_mutation.h
index 28dec1c8a..dfbac666d 100644
--- a/src/include/utils/pool_track_table_mutation.h
+++ b/src/include/utils/pool_track_table_mutation.h
@@ -26,17 +26,6 @@
#include "pool.h"
#include <sys/time.h>
-/*
- * Maximum table name length including schema: "schema"."table"
- * Using NAMEDATALEN * 2 + 4 for quotes and dot
- */
-#define TRACK_TABLE_MUTATION_TABLE_NAME_LEN (NAMEDATALEN * 2 + 4)
-
-/*
- * Maximum number of tables we track per query
- */
-#define TRACK_TABLE_MUTATION_MAX_TABLES_PER_QUERY 8
-
/*
* Invalid index marker for linked lists
*/
@@ -77,41 +66,6 @@ typedef struct TrackTableMutationHashTable
*/
} TrackTableMutationHashTable;
-/*
- * Entry in the query parse cache
- */
-typedef struct QueryParseEntry
-{
- uint64 query_hash; /* Hash of normalized query */
- bool is_write; /* True if INSERT/UPDATE/DELETE */
- int num_tables; /* Number of tables in query */
- char table_names
- [ TRACK_TABLE_MUTATION_MAX_TABLES_PER_QUERY]
- [ TRACK_TABLE_MUTATION_TABLE_NAME_LEN];
- int next; /* Next entry in collision chain */
- int lru_prev; /* Previous in LRU list */
- int lru_next; /* Next in LRU list */
- bool in_use; /* Is this entry in use? */
-} QueryParseEntry;
-
-/*
- * Header for the query parse cache in shared memory
- */
-typedef struct QueryParseCache
-{
- int num_buckets; /* Number of hash buckets */
- int max_entries; /* Maximum entries allowed */
- int num_entries; /* Current number of entries */
- int free_list_head; /* Head of free entry list */
- int lru_head; /* Most recently used */
- int lru_tail; /* Least recently used */
-
- /*
- * Flexible array members follow in shared memory: int
- * buckets[num_buckets]; QueryParseEntry entries[max_entries];
- */
-} QueryParseCache;
-
/*
* Global state for track table mutation feature
*/
@@ -134,7 +88,6 @@ typedef struct TrackTableMutationShmem
{
TrackTableMutationState state;
TrackTableMutationHashTable *table_map;
- QueryParseCache *query_cache;
} TrackTableMutationShmem;
/* ----------------
@@ -206,39 +159,6 @@ extern void pool_track_table_mutation_mark_table_written(
*/
extern void pool_track_table_mutation_update_ttl(uint64 delay_us);
-/*
- * Look up cached parse result for a query.
- * hash: hash of normalized query
- * is_write: output - true if query is a write
- * table_names: output - array to fill with table names
- * num_tables: output - number of tables found
- * Returns true if found in cache, false otherwise.
- */
-extern bool pool_track_table_mutation_get_cached_parse(
- uint64 hash, bool *is_write,
- char table_names[][TRACK_TABLE_MUTATION_TABLE_NAME_LEN],
- int *num_tables);
-
-/*
- * Cache a parse result for a query.
- * hash: hash of normalized query
- * is_write: true if query is a write
- * table_names: array of table names
- * num_tables: number of tables
- */
-extern void pool_track_table_mutation_cache_parse(
- uint64 hash, bool is_write,
- const char table_names[][TRACK_TABLE_MUTATION_TABLE_NAME_LEN],
- int num_tables);
-
-/*
- * Normalize a query and compute its hash.
- * Strips comments, normalizes whitespace and literals.
- * query: input SQL query string
- * Returns: 64-bit hash of normalized query
- */
-extern uint64 pool_track_table_mutation_normalize_and_hash(const char *query);
-
/*
* Calculate required shared memory size for track table mutation.
*/
diff --git a/src/sample/pgpool.conf.sample-stream b/src/sample/pgpool.conf.sample-stream
index 00132d534..ce9b92da0 100644
--- a/src/sample/pgpool.conf.sample-stream
+++ b/src/sample/pgpool.conf.sample-stream
@@ -509,8 +509,7 @@ backend_clustering_mode = streaming_replication
# - Track Table Mutation (used by dml_adaptive_global) -
# WARNING: dml_adaptive_global increases shared memory usage
- # Default settings require ~6.4 MB shared memory
- # (0.1 MB table tracking + 6.3 MB query cache)
+ # Default settings require ~80 KB shared memory for table tracking
#track_table_mutation_ttl_factor = 5.0
# TTL multiplier: TTL = replication_delay * factor
@@ -544,16 +543,6 @@ backend_clustering_mode = streaming_replication
# Range: 128-131072 (default: 2048)
# (change requires restart)
-#track_table_mutation_query_buckets = 2048
- # Number of hash buckets for query parse cache
- # Range: 64-65536 (default: 2048)
- # (change requires restart)
-
-#track_table_mutation_query_parse_cache_size = 10000
- # Maximum number of query parse results to cache
- # Range: 100-1000000 (default: 10000)
- # Memory usage: ~640 bytes per entry (~6.3 MB default, ~64 MB for 100000)
- # (change requires restart)
#------------------------------------------------------------------------------
# STREAMING REPLICATION MODE
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 0f1fa884c..467ec114c 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -431,8 +431,6 @@ PublicationObjSpec
PublicationObjSpecType
PublicationTable
Query
-QueryParseCache
-QueryParseEntry
QuerySource
RELQTARGET_OPTION
RTEKind
diff --git a/src/utils/pool_track_table_mutation.c b/src/utils/pool_track_table_mutation.c
index 9be46b28f..e7771e7bf 100644
--- a/src/utils/pool_track_table_mutation.c
+++ b/src/utils/pool_track_table_mutation.c
@@ -76,16 +76,6 @@ static bool cold_start_initialized = false;
sizeof(TrackTableMutationHashTable) + \
(map)->num_buckets * sizeof(int)))
-/* Get pointer to bucket array in parse cache */
-#define PARSE_CACHE_BUCKETS(cache) \
- ((int *)((char *)(cache) + sizeof(QueryParseCache)))
-
-/* Get pointer to entry array in parse cache */
-#define PARSE_CACHE_ENTRIES(cache) \
- ((QueryParseEntry *)((char *)(cache) + \
- sizeof(QueryParseCache) + \
- (cache)->num_buckets * sizeof(int)))
-
/* ----------------
* Semaphore lock helpers
* ----------------
@@ -103,18 +93,6 @@ table_map_unlock(void)
pool_semaphore_unlock(TRACK_TABLE_MUTATION_TABLE_SEM);
}
-static inline void
-parse_cache_lock(void)
-{
- pool_semaphore_lock(TRACK_TABLE_MUTATION_QUERY_SEM);
-}
-
-static inline void
-parse_cache_unlock(void)
-{
- pool_semaphore_unlock(TRACK_TABLE_MUTATION_QUERY_SEM);
-}
-
/* ----------------
* Hash functions
* ----------------
@@ -144,25 +122,6 @@ fnv1a_hash_table_key(int table_oid, int dboid)
return hash;
}
-/*
- * FNV-1a hash for 64-bit value
- */
-static uint64
-fnv1a_hash_64(const char *str, size_t len)
-{
- /* FNV offset basis for 64-bit */
- uint64 hash = 14695981039346656037ULL;
- size_t i;
-
- for (i = 0; i < len; i++)
- {
- hash ^= (uint8) str[i];
- hash *= 1099511628211ULL; /* FNV prime */
- }
-
- return hash;
-}
-
/* ----------------
* Time utilities
* ----------------
@@ -514,334 +473,6 @@ table_map_cleanup_expired(
}
}
-/* ----------------
- * Parse cache operations
- * ----------------
- */
-
-/*
- * Initialize parse cache
- */
-static void
-parse_cache_init(QueryParseCache * cache,
- int num_buckets, int max_entries)
-{
- int *buckets;
- QueryParseEntry *entries;
- int i;
- int invalid = TRACK_TABLE_MUTATION_INVALID_INDEX;
-
- cache->num_buckets = num_buckets;
- cache->max_entries = max_entries;
- cache->num_entries = 0;
- cache->free_list_head = 0;
- cache->lru_head = invalid;
- cache->lru_tail = invalid;
-
- buckets = PARSE_CACHE_BUCKETS(cache);
- entries = PARSE_CACHE_ENTRIES(cache);
-
- /* Initialize all buckets to empty */
- for (i = 0; i < num_buckets; i++)
- buckets[i] = invalid;
-
- /* Initialize free list */
- for (i = 0; i < max_entries; i++)
- {
- entries[i].in_use = false;
- entries[i].next = (i < max_entries - 1) ?
- i + 1 : invalid;
- entries[i].lru_prev = invalid;
- entries[i].lru_next = invalid;
- }
-
- ereport(DEBUG1,
- (errmsg("track_table_mutation: "
- "parse cache init %d buckets, "
- "%d max entries",
- num_buckets, max_entries)));
-}
-
-/*
- * Move entry to front of LRU list (most recently used)
- */
-static void
-parse_cache_lru_touch(QueryParseCache * cache, int idx)
-{
- QueryParseEntry *entries = PARSE_CACHE_ENTRIES(cache);
- int invalid = TRACK_TABLE_MUTATION_INVALID_INDEX;
-
- /* Already at head? */
- if (cache->lru_head == idx)
- return;
-
- /* Remove from current position */
- if (entries[idx].lru_prev != invalid)
- entries[entries[idx].lru_prev].lru_next =
- entries[idx].lru_next;
- if (entries[idx].lru_next != invalid)
- entries[entries[idx].lru_next].lru_prev =
- entries[idx].lru_prev;
- if (cache->lru_tail == idx)
- cache->lru_tail = entries[idx].lru_prev;
-
- /* Insert at head */
- entries[idx].lru_prev = invalid;
- entries[idx].lru_next = cache->lru_head;
- if (cache->lru_head != invalid)
- entries[cache->lru_head].lru_prev = idx;
- cache->lru_head = idx;
- if (cache->lru_tail == invalid)
- cache->lru_tail = idx;
-}
-
-/*
- * Add entry to LRU list (at head)
- */
-static void
-parse_cache_lru_add(QueryParseCache * cache, int idx)
-{
- QueryParseEntry *entries = PARSE_CACHE_ENTRIES(cache);
- int invalid = TRACK_TABLE_MUTATION_INVALID_INDEX;
-
- entries[idx].lru_prev = invalid;
- entries[idx].lru_next = cache->lru_head;
-
- if (cache->lru_head != invalid)
- entries[cache->lru_head].lru_prev = idx;
-
- cache->lru_head = idx;
-
- if (cache->lru_tail == invalid)
- cache->lru_tail = idx;
-}
-
-/*
- * Remove entry from LRU list
- */
-static void
-parse_cache_lru_remove(QueryParseCache * cache, int idx)
-{
- QueryParseEntry *entries = PARSE_CACHE_ENTRIES(cache);
- int invalid = TRACK_TABLE_MUTATION_INVALID_INDEX;
-
- if (entries[idx].lru_prev != invalid)
- entries[entries[idx].lru_prev].lru_next =
- entries[idx].lru_next;
- else
- cache->lru_head = entries[idx].lru_next;
-
- if (entries[idx].lru_next != invalid)
- entries[entries[idx].lru_next].lru_prev =
- entries[idx].lru_prev;
- else
- cache->lru_tail = entries[idx].lru_prev;
-
- entries[idx].lru_prev = invalid;
- entries[idx].lru_next = invalid;
-}
-
-/*
- * Allocate entry from free list, evicting LRU if needed
- */
-static int
-parse_cache_alloc_entry(QueryParseCache * cache)
-{
- QueryParseEntry *entries = PARSE_CACHE_ENTRIES(cache);
- int *buckets = PARSE_CACHE_BUCKETS(cache);
- int idx;
- int invalid = TRACK_TABLE_MUTATION_INVALID_INDEX;
-
- if (cache->free_list_head != invalid)
- {
- idx = cache->free_list_head;
- cache->free_list_head = entries[idx].next;
- entries[idx].in_use = true;
- entries[idx].next = invalid;
- cache->num_entries++;
- return idx;
- }
-
- /* No free entries - evict LRU */
- if (cache->lru_tail == invalid)
- return invalid;
-
- idx = cache->lru_tail;
-
- /* Remove from hash bucket */
- {
- int bucket;
- int *prev_ptr;
- int curr;
-
- bucket = entries[idx].query_hash %
- cache->num_buckets;
- prev_ptr = &buckets[bucket];
- curr = buckets[bucket];
-
- while (curr != invalid)
- {
- if (curr == idx)
- {
- *prev_ptr = entries[curr].next;
- break;
- }
- prev_ptr = &entries[curr].next;
- curr = entries[curr].next;
- }
- }
-
- /* Remove from LRU list */
- parse_cache_lru_remove(cache, idx);
-
- /* Reinitialize entry */
- entries[idx].in_use = true;
- entries[idx].next = invalid;
-
- return idx;
-}
-
-/*
- * Look up a query in the parse cache
- */
-static int
-parse_cache_lookup(QueryParseCache * cache, uint64 hash)
-{
- int *buckets = PARSE_CACHE_BUCKETS(cache);
- QueryParseEntry *entries = PARSE_CACHE_ENTRIES(cache);
- int bucket = hash % cache->num_buckets;
- int idx = buckets[bucket];
- int invalid = TRACK_TABLE_MUTATION_INVALID_INDEX;
-
- while (idx != invalid)
- {
- if (entries[idx].query_hash == hash)
- return idx;
- idx = entries[idx].next;
- }
-
- return invalid;
-}
-
-/* ----------------
- * Query normalization
- * ----------------
- */
-
-/*
- * Simple query normalization:
- * - Strip comments (-- and C-style block comments)
- * - Collapse whitespace
- * - Convert to lowercase (except inside strings)
- * - Replace literal values with placeholders
- */
-static size_t
-normalize_query(const char *query, char *output,
- size_t output_size)
-{
- const char *src = query;
- char *dst = output;
- char *dst_end = output + output_size - 1;
- bool in_string = false;
- char string_char = 0;
- bool last_was_space = true;
-
- while (*src && dst < dst_end)
- {
- /* Handle string literals */
- if (in_string)
- {
- if (*src == string_char)
- {
- if (*(src + 1) == string_char)
- {
- /* Escaped quote */
- src += 2;
- continue;
- }
- in_string = false;
- /* Replace string with placeholder */
- *dst++ = '$';
- }
- src++;
- continue;
- }
-
- /* Check for string start */
- if (*src == '\'' || *src == '"')
- {
- in_string = true;
- string_char = *src;
- src++;
- continue;
- }
-
- /* Handle single-line comments */
- if (*src == '-' && *(src + 1) == '-')
- {
- while (*src && *src != '\n')
- src++;
- continue;
- }
-
- /* Handle multi-line comments */
- if (*src == '/' && *(src + 1) == '*')
- {
- src += 2;
- while (*src &&
- !(*src == '*' && *(src + 1) == '/'))
- src++;
- if (*src)
- src += 2;
- continue;
- }
-
- /* Handle whitespace */
- if (*src == ' ' || *src == '\t' ||
- *src == '\n' || *src == '\r')
- {
- if (!last_was_space)
- {
- *dst++ = ' ';
- last_was_space = true;
- }
- src++;
- continue;
- }
-
- /* Handle numbers - replace with placeholder */
- if ((*src >= '0' && *src <= '9') ||
- (*src == '.' && *(src + 1) >= '0' &&
- *(src + 1) <= '9'))
- {
- while (*src &&
- ((*src >= '0' && *src <= '9') ||
- *src == '.'))
- src++;
- if (!last_was_space &&
- dst > output && *(dst - 1) != '$')
- *dst++ = '$';
- last_was_space = false;
- continue;
- }
-
- /* Regular character - convert to lowercase */
- if (*src >= 'A' && *src <= 'Z')
- *dst++ = *src + 32;
- else
- *dst++ = *src;
-
- last_was_space = false;
- src++;
- }
-
- /* Remove trailing space */
- if (dst > output && *(dst - 1) == ' ')
- dst--;
-
- *dst = '\0';
- return dst - output;
-}
/* ----------------
* Public API implementation
@@ -858,13 +489,9 @@ pool_track_table_mutation_shmem_size(void)
Size size = 0;
int tbl_bkt;
int tbl_sz;
- int qry_bkt;
- int qry_sz;
tbl_bkt = pool_config->track_table_mutation_table_buckets;
tbl_sz = pool_config->track_table_mutation_table_size;
- qry_bkt = pool_config->track_table_mutation_query_buckets;
- qry_sz = pool_config->track_table_mutation_query_parse_cache_size;
/* Main structure */
size += sizeof(TrackTableMutationShmem);
@@ -874,11 +501,6 @@ pool_track_table_mutation_shmem_size(void)
size += tbl_bkt * sizeof(int);
size += tbl_sz * sizeof(TrackTableMutationEntry);
- /* Parse cache */
- size += sizeof(QueryParseCache);
- size += qry_bkt * sizeof(int);
- size += qry_sz * sizeof(QueryParseEntry);
-
return size;
}
@@ -897,8 +519,6 @@ pool_track_table_mutation_init(void)
TrackTableMutationState *st;
int tbl_bkt;
int tbl_sz;
- int qry_bkt;
- int qry_sz;
if (pool_config->disable_load_balance_on_write !=
DLBOW_DML_ADAPTIVE_GLOBAL)
@@ -911,8 +531,6 @@ pool_track_table_mutation_init(void)
tbl_bkt = pool_config->track_table_mutation_table_buckets;
tbl_sz = pool_config->track_table_mutation_table_size;
- qry_bkt = pool_config->track_table_mutation_query_buckets;
- qry_sz = pool_config->track_table_mutation_query_parse_cache_size;
shmem_size = pool_track_table_mutation_shmem_size();
@@ -938,22 +556,12 @@ pool_track_table_mutation_init(void)
track_table_mutation_shmem->table_map =
(TrackTableMutationHashTable *) shmem_ptr;
- shmem_ptr += sizeof(TrackTableMutationHashTable);
- shmem_ptr += tbl_bkt * sizeof(int);
- shmem_ptr += tbl_sz * sizeof(TrackTableMutationEntry);
- track_table_mutation_shmem->query_cache =
- (QueryParseCache *) shmem_ptr;
-
- /* Initialize structures */
+ /* Initialize table map */
table_map_init(
track_table_mutation_shmem->table_map,
tbl_bkt, tbl_sz);
- parse_cache_init(
- track_table_mutation_shmem->query_cache,
- qry_bkt, qry_sz);
-
/* Initialize global state */
st = &track_table_mutation_shmem->state;
st->initialized = true;
@@ -1292,159 +900,3 @@ pool_track_table_mutation_update_ttl(uint64 delay_us)
(unsigned long) delay_us,
factor)));
}
-
-/*
- * Look up a cached parse result by query hash.
- * Returns true and fills output parameters if
- * the query was found in the parse cache.
- */
-bool
-pool_track_table_mutation_get_cached_parse(
- uint64 hash, bool *is_write,
- char table_names[][TRACK_TABLE_MUTATION_TABLE_NAME_LEN],
- int *num_tables)
-{
- QueryParseCache *cache;
- int idx;
- bool found = false;
- int max_tables;
-
- if (TRACK_TABLE_MUTATION_DISABLED())
- return false;
-
- max_tables = TRACK_TABLE_MUTATION_MAX_TABLES_PER_QUERY;
- cache = track_table_mutation_shmem->query_cache;
-
- parse_cache_lock();
-
- idx = parse_cache_lookup(cache, hash);
- if (idx != TRACK_TABLE_MUTATION_INVALID_INDEX)
- {
- QueryParseEntry *entries;
- int i;
- int namelen;
-
- entries = PARSE_CACHE_ENTRIES(cache);
- namelen = TRACK_TABLE_MUTATION_TABLE_NAME_LEN;
- *is_write = entries[idx].is_write;
- *num_tables = entries[idx].num_tables;
-
- for (i = 0;
- i < entries[idx].num_tables &&
- i < max_tables;
- i++)
- {
- strlcpy(table_names[i],
- entries[idx].table_names[i],
- namelen);
- }
-
- /* Move to front of LRU */
- parse_cache_lru_touch(cache, idx);
- found = true;
- }
-
- parse_cache_unlock();
-
- return found;
-}
-
-/*
- * Store a parse result in the shared cache.
- * Evicts the LRU entry if the cache is full.
- */
-void
-pool_track_table_mutation_cache_parse(
- uint64 hash, bool is_write,
- const char table_names[][TRACK_TABLE_MUTATION_TABLE_NAME_LEN],
- int num_tables)
-{
- QueryParseCache *cache;
- int *buckets;
- QueryParseEntry *entries;
- int idx;
- int bucket;
- int max_tables;
- int namelen;
-
- if (TRACK_TABLE_MUTATION_DISABLED())
- return;
-
- max_tables = TRACK_TABLE_MUTATION_MAX_TABLES_PER_QUERY;
- namelen = TRACK_TABLE_MUTATION_TABLE_NAME_LEN;
- cache = track_table_mutation_shmem->query_cache;
-
- parse_cache_lock();
-
- /* Check if already exists */
- idx = parse_cache_lookup(cache, hash);
- if (idx != TRACK_TABLE_MUTATION_INVALID_INDEX)
- {
- parse_cache_unlock();
- return;
- }
-
- /* Allocate new entry (may evict LRU) */
- idx = parse_cache_alloc_entry(cache);
- if (idx == TRACK_TABLE_MUTATION_INVALID_INDEX)
- {
- parse_cache_unlock();
- ereport(WARNING,
- (errmsg("track_table_mutation: "
- "parse cache alloc failed")));
- return;
- }
-
- entries = PARSE_CACHE_ENTRIES(cache);
- buckets = PARSE_CACHE_BUCKETS(cache);
-
- /* Fill in entry */
- entries[idx].query_hash = hash;
- entries[idx].is_write = is_write;
- entries[idx].num_tables =
- (num_tables > max_tables) ?
- max_tables : num_tables;
-
- {
- int i;
-
- for (i = 0; i < entries[idx].num_tables; i++)
- {
- strlcpy(entries[idx].table_names[i],
- table_names[i], namelen);
- }
- }
-
- /* Insert into hash bucket */
- bucket = hash % cache->num_buckets;
- entries[idx].next = buckets[bucket];
- buckets[bucket] = idx;
-
- /* Add to LRU list */
- parse_cache_lru_add(cache, idx);
-
- parse_cache_unlock();
-}
-
-/*
- * Normalize a SQL query and compute its 64-bit hash.
- * Strips comments, collapses whitespace, lowercases,
- * and replaces literals with placeholders.
- */
-uint64
-pool_track_table_mutation_normalize_and_hash(
- const char *query)
-{
- char normalized[8192];
- size_t len;
-
- if (query == NULL || query[0] == '\0')
- return 0;
-
- len = normalize_query(query, normalized,
- sizeof(normalized));
- if (len == 0)
- return 0;
-
- return fnv1a_hash_64(normalized, len);
-}
--
2.53.0
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: Proposal: Recent mutated table tracking in memory
In-Reply-To: <CACeKOO3k8K0=u9AKmctyFVivrCuMtN5OdcCDXt0ew=qwbrQGcA@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox