pg.ddx.io pgdogdev/pgdog GitHub issues and pull requests
help / color / mirror / Atom feed[pgdogdev/pgdog] PR #2072: refactor(cross-shard): sharding key lookup for aggregate rewrite
5+ messages / 4 participants
[nested] [flat]
* [pgdogdev/pgdog] PR #2072: refactor(cross-shard): sharding key lookup for aggregate rewrite
@ 2026-11-19 13:21 "murex971 (@murex971)" <noreply+murex971@github.com>
0 siblings, 0 replies; 5+ messages in thread
From: murex971 (@murex971) @ 2026-11-19 13:21 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
### Summary
Groundwork for cross-shard `DISTINCT` aggregates. No behaviour change.
When `DISTINCT` is on the sharding key, or `GROUP BY` includes it, shards
never share a value or a group, so per-shard results can be added up.
The aggregate rewrite needs to know when that is the case.
- Move the router's sharding key lookup to `ShardedTables::sharding_key`
so the rewrite can reuse it. The router calls the same method.
- Add `ShardingKey`, which answers whether an expression or the
`GROUP BY` is the sharding key of the table a `SELECT` reads. It only
answers for a single plain table and a bare column reference; joins,
subqueries, CTEs and expressions over the key are treated as "no".
inline patch 1: pgdog/src/backend/replication/sharded_tables.rs, pgdog/src/frontend/router/parser/rewrite/statement/aggregate/mod.rs, pgdog/src/frontend/router/parser/rewrite/statement/aggregate/sharding_key.rs and 1 more (+262/-26 lines) download inline-1.patch
diff --git a/pgdog/src/backend/replication/sharded_tables.rs b/pgdog/src/backend/replication/sharded_tables.rs
index b8cbb358f..46b387e61 100644
--- a/pgdog/src/backend/replication/sharded_tables.rs
+++ b/pgdog/src/backend/replication/sharded_tables.rs
@@ -247,6 +247,32 @@ impl ShardedTables {
None
}
+
+ /// Sharded table config for a column, if the column is a sharding key.
+ ///
+ /// Configs that name a table match that table's column. Configs with only
+ /// a column match that column on any table.
+ pub(crate) fn sharding_key(&self, column: Column<'_>) -> Option<&ShardedTable> {
+ // Omnisharded config takes priority over sharded tables:
+ // a sharding key on an omnisharded table doesn't route.
+ if column
+ .table
+ .is_some_and(|name| self.omnishards().contains_key(name))
+ {
+ return None;
+ }
+
+ if let Some(table) = self.get_table(column)
+ && table.name.is_some()
+ {
+ return Some(table);
+ }
+
+ // Column-only config: user explicitly wants any table with this column to be sharded
+ self.tables()
+ .iter()
+ .find(|table| table.name.is_none() && table.column == column.name)
+ }
}
#[cfg(test)]
diff --git a/pgdog/src/frontend/router/parser/rewrite/statement/aggregate/mod.rs b/pgdog/src/frontend/router/parser/rewrite/statement/aggregate/mod.rs
index 5709b4589..119a8f468 100644
--- a/pgdog/src/frontend/router/parser/rewrite/statement/aggregate/mod.rs
+++ b/pgdog/src/frontend/router/parser/rewrite/statement/aggregate/mod.rs
@@ -1,4 +1,7 @@
mod engine;
+// Used by the DISTINCT aggregate rewrite.
+#[cfg_attr(not(test), expect(dead_code))]
+mod sharding_key;
pub(crate) use super::projection::AggregateHelper;
diff --git a/pgdog/src/frontend/router/parser/rewrite/statement/aggregate/sharding_key.rs b/pgdog/src/frontend/router/parser/rewrite/statement/aggregate/sharding_key.rs
new file mode 100644
index 000000000..83f358d52
--- /dev/null
+++ b/pgdog/src/frontend/router/parser/rewrite/statement/aggregate/sharding_key.rs
@@ -0,0 +1,228 @@
+//! Sharding key of the table a SELECT reads.
+//!
+//! A sharding key value lives on one shard, so shards never share one. Each
+//! shard's `DISTINCT` aggregate over the key covers values no other shard
+//! has, and a group that includes the key is complete on its shard.
+
+use itertools::Itertools;
+use pg_raw_parse::{Node, nodes};
+
+use crate::backend::replication::ShardedTables;
+use crate::frontend::router::parser::{Column, Table};
+
+/// Answers whether an expression is the sharding key of the queried table.
+pub(crate) struct ShardingKey<'a> {
+ table: Table<'a>,
+ tables: &'a ShardedTables,
+}
+
+impl<'a> ShardingKey<'a> {
+ /// `None` unless the SELECT reads exactly one plain table. Joins,
+ /// subqueries and functions in `FROM` can repeat a key on several shards,
+ /// and a CTE can shadow the table's name.
+ pub(crate) fn new(select: &'a nodes::SelectStmt, tables: &'a ShardedTables) -> Option<Self> {
+ if select.with_clause().is_some() {
+ return None;
+ }
+ let Ok(Node::RangeVar(range_var)) = select.from_clause().iter().exactly_one() else {
+ return None;
+ };
+
+ Some(Self {
+ table: Table::from(range_var),
+ tables,
+ })
+ }
+
+ /// Whether `node` is a bare reference to the sharding key column.
+ /// Expressions over the key, e.g. `user_id % 2`, don't count:
+ /// two shards can produce the same result.
+ pub(crate) fn matches(&self, node: Node<'_>) -> bool {
+ let Node::ColumnRef(column) = node else {
+ return false;
+ };
+ let Ok(mut column) = Column::try_from(column) else {
+ return false;
+ };
+ if !self.qualifier_matches(&column) {
+ return false;
+ }
+ column.table = Some(self.table.name);
+ column.schema = self.table.schema;
+
+ self.tables.sharding_key(column).is_some()
+ }
+
+ /// Whether `GROUP BY` includes the sharding key, by name or by SELECT
+ /// list position, so that no group spans shards.
+ pub(crate) fn in_group_by(&self, select: &nodes::SelectStmt) -> bool {
+ select.group_clause().iter().any(|item| match item {
+ // Postgres counts SELECT list positions from 1.
+ Node::A_Const(ordinal) => ordinal
+ .val()
+ .and_then(|value| value.numeric_value::<i32>())
+ .and_then(|ordinal| usize::try_from(ordinal).ok()?.checked_sub(1))
+ .and_then(|position| select.target_list().iter().nth(position))
+ .is_some_and(|target| self.matches(target.val())),
+ item => self.matches(item),
+ })
+ }
+
+ /// An aliased table is only known by its alias.
+ fn qualifier_matches(&self, column: &Column<'_>) -> bool {
+ match (column.schema, column.table, self.table.alias) {
+ (None, None, _) => true,
+ (None, Some(qualifier), Some(alias)) => qualifier == alias,
+ (None, Some(qualifier), None) => qualifier == self.table.name,
+ (Some(schema), Some(qualifier), None) => {
+ qualifier == self.table.name && self.table.schema == Some(schema)
+ }
+ _ => false,
+ }
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use crate::frontend::router::sharding::ShardedTable;
+ use pg_raw_parse::{Owned, make};
+ use pgdog_config::{OmnishardedTable, SystemCatalogsBehavior};
+
+ fn select(sql: &str) -> Owned<nodes::SelectStmt> {
+ match pg_raw_parse::parse(sql)
+ .expect("valid SQL")
+ .stmts()
+ .next()
+ .expect("one statement")
+ {
+ Node::SelectStmt(stmt) => make::owned(|mem| mem.make_unique(stmt)),
+ _ => panic!("not a select"),
+ }
+ }
+
+ fn config(name: Option<&str>, column: &str) -> ShardedTable {
+ ShardedTable {
+ name: name.map(Into::into),
+ column: column.into(),
+ ..Default::default()
+ }
+ }
+
+ fn orders_by_user() -> ShardedTables {
+ ShardedTables::from([config(Some("orders"), "user_id")].as_slice())
+ }
+
+ /// Whether the first SELECT list item and the GROUP BY are on the sharding key.
+ fn check(sql: &str, tables: &ShardedTables) -> Option<(bool, bool)> {
+ let select = select(sql);
+ let key = ShardingKey::new(&select, tables)?;
+ let first = select.target_list().iter().next().expect("one target");
+
+ Some((key.matches(first.val()), key.in_group_by(&select)))
+ }
+
+ #[test]
+ fn matches_bare_key_column() {
+ let tables = orders_by_user();
+
+ for (sql, expected) in [
+ ("SELECT user_id FROM orders", true),
+ ("SELECT orders.user_id FROM orders", true),
+ ("SELECT public.orders.user_id FROM public.orders", true),
+ ("SELECT o.user_id FROM orders o", true),
+ ("SELECT product_id FROM orders", false),
+ // Two shards can produce the same value.
+ ("SELECT user_id % 2 FROM orders", false),
+ ("SELECT user_id::text FROM orders", false),
+ // The alias hides the table name.
+ ("SELECT orders.user_id FROM orders o", false),
+ ("SELECT other.user_id FROM orders", false),
+ ("SELECT other.orders.user_id FROM public.orders", false),
+ ("SELECT user_id FROM users", false),
+ ] {
+ assert_eq!(
+ check(sql, &tables).map(|(first, _)| first),
+ Some(expected),
+ "{sql}"
+ );
+ }
+ }
+
+ #[test]
+ fn requires_one_plain_table() {
+ let tables = orders_by_user();
+
+ for sql in [
+ "SELECT user_id FROM orders JOIN users ON users.id = orders.user_id",
+ "SELECT user_id FROM orders, users",
+ "SELECT user_id FROM (SELECT user_id FROM orders) AS orders",
+ "SELECT user_id FROM generate_series(1, 3) AS orders(user_id)",
+ "WITH orders AS (SELECT 1 AS user_id) SELECT user_id FROM orders",
+ "SELECT user_id FROM orders UNION SELECT user_id FROM orders",
+ "SELECT 1",
+ ] {
+ assert!(check(sql, &tables).is_none(), "{sql}");
+ }
+ }
+
+ #[test]
+ fn finds_key_in_group_by() {
+ let tables = orders_by_user();
+
+ for (sql, expected) in [
+ ("SELECT count(*) FROM orders GROUP BY user_id", true),
+ (
+ "SELECT count(*) FROM orders GROUP BY status, o.user_id",
+ false,
+ ),
+ (
+ "SELECT count(*) FROM orders o GROUP BY status, o.user_id",
+ true,
+ ),
+ (
+ "SELECT status, user_id, count(*) FROM orders GROUP BY 1, 2",
+ true,
+ ),
+ ("SELECT status, count(*) FROM orders GROUP BY 1", false),
+ ("SELECT count(*) FROM orders GROUP BY status", false),
+ ("SELECT count(*) FROM orders GROUP BY user_id % 2", false),
+ ("SELECT count(*) FROM orders", false),
+ // The grand total group spans shards.
+ (
+ "SELECT count(*) FROM orders GROUP BY ROLLUP (user_id)",
+ false,
+ ),
+ ] {
+ assert_eq!(
+ check(sql, &tables).map(|(_, group)| group),
+ Some(expected),
+ "{sql}"
+ );
+ }
+ }
+
+ #[test]
+ fn follows_sharded_table_config() {
+ // A config without a table name covers the column on every table.
+ let column_only = ShardedTables::from([config(None, "user_id")].as_slice());
+ assert_eq!(
+ check("SELECT user_id FROM anything", &column_only),
+ Some((true, false))
+ );
+
+ let omnisharded = ShardedTables::new(
+ vec![config(None, "user_id")],
+ vec![OmnishardedTable {
+ name: "settings".into(),
+ sticky_routing: false,
+ }],
+ false,
+ SystemCatalogsBehavior::default(),
+ );
+ assert_eq!(
+ check("SELECT user_id FROM settings", &omnisharded),
+ Some((false, false))
+ );
+ }
+}
diff --git a/pgdog/src/frontend/router/parser/statement.rs b/pgdog/src/frontend/router/parser/statement.rs
index 74fe69df8..e810b0062 100644
--- a/pgdog/src/frontend/router/parser/statement.rs
+++ b/pgdog/src/frontend/router/parser/statement.rs
@@ -660,32 +660,11 @@ impl<'a, 'b: 'a> StatementParser<'a, 'b> {
table_name: Option<&str>,
schema: Option<&str>,
) -> Option<&'b ShardedTable> {
- // Omnisharded config takes priority over sharded tables:
- // a sharding key on an omnisharded table doesn't route.
- if table_name.is_some_and(|name| self.schema.tables.omnishards().contains_key(name)) {
- return None;
- }
-
- // Try named table configs first
- if let Some(table_name) = table_name {
- let column = Column {
- name: column_name,
- table: Some(table_name),
- schema,
- };
- if let Some(sharded_table) = self.schema.tables().get_table(column)
- && sharded_table.name.is_some()
- {
- return Some(sharded_table);
- }
- }
-
- // Column-only config: user explicitly wants any table with this column to be sharded
- self.schema
- .tables
- .tables()
- .iter()
- .find(|t| t.name.is_none() && t.column == column_name)
+ self.schema.tables().sharding_key(Column {
+ name: column_name,
+ table: table_name,
+ schema,
+ })
}
fn compute_shard(
^ permalink raw [nested|flat] 5+ messages in thread
* Re: [pgdogdev/pgdog] PR #2072: refactor(cross-shard): sharding key lookup for aggregate rewrite
@ 2026-11-19 13:37 "codecov[bot] (bot) (@codecov[bot])" <noreply+codecov[bot]@github.com>
3 siblings, 0 replies; 5+ messages in thread
From: codecov[bot] (bot) (@codecov[bot]) @ 2026-11-19 13:37 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
## [Codecov](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2072?dropdown=coverage&src=pr&el=h1&utm_me...) Report
:x: Patch coverage is `98.18182%` with `3 lines` in your changes missing coverage. Please review.
| [Files with missing lines](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2072?dropdown=coverage&src=pr&el=tree&utm_...) | Patch % | Lines |
|---|---|---|
| [...parser/rewrite/statement/aggregate/sharding\_key.rs](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2072?src=pr&el=tree&utm_medium=referral&ut...) | 97.94% | [3 Missing :warning: ](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2072?src=pr&el=tree&utm_medium=referral&ut...) |
:loudspeaker: Thoughts on this report? [Let us know!](https://github.com/codecov/feedback/issues/255)
^ permalink raw [nested|flat] 5+ messages in thread
* Re: [pgdogdev/pgdog] PR #2072: refactor(cross-shard): sharding key lookup for aggregate rewrite
@ 2026-11-19 14:03 "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
3 siblings, 0 replies; 5+ messages in thread
From: github-actions[bot] (bot) (@github-actions[bot]) @ 2026-11-19 14:03 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
<!-- metal-bench -->
**Metal bench (arm64) passed.** All 520 measures within bounds.
| suite | throughput | cpu/txn | user cycles/txn | user instr/txn | kernel cycles/txn | kernel instr/txn | ctx/txn |
|---|---|---|---|---|---|---|---|
| pooler | | 19.03 (+0.3% ± 1.2%) | 18,068 (-0.0% ± 0.8%) | 31,448 (-0.0% ± 0.04%) | 30,642 (+0.6% ± 2.1%) | 51,112 (+0.0% ± 0.4%) | |
| lb | | 19.59 (+0.5% ± 1.6%) | 18,653 (+0.2% ± 1.0%) | 32,348 (-0.0% ± 0.05%) | 31,300 (+1.1% ± 2.3%) | 52,228 (+0.2% ± 0.5%) | |
| sharding-w2 | | 32.28 (-0.2% ± 1.7%) | 43,386 (-0.2% ± 0.9%) | 94,901 (+0.0% ± 0.02%) | 40,460 (-0.2% ± 3.0%) | 53,742 (+0.1% ± 0.1%) | 1.547 (+0.1% ± 0.1%) |
| sharding-w8 | 243,532 (+0.5% ± 3.5%) | 29.43 (-0.3% ± 3.3%) | 44,253 (+0.1% ± 3.0%) | 50,494 (-0.0% ± 0.09%) | 37,021 (-0.8% ± 3.7%) | 42,373 (-0.0% ± 0.09%) | 0.166 (-1.0% ± 0.8%) |
<details><summary>Counters per txn</summary>
| suite | frontend stall | l1i refill | backend stall |
|---|---|---|---|
| pooler | 7,789 (+0.0% ± 2.1%) | 2,677 (-0.1% ± 0.9%) | |
| lb | 8,142 (+0.7% ± 2.4%) | 2,728 (+0.2% ± 1.1%) | |
| sharding-w2 | 17,981 (-0.3% ± 1.8%) | 4,167 (-0.2% ± 0.9%) | |
| sharding-w8 | 20,411 (+1.0% ± 5.0%) | 4,184 (+0.4% ± 2.5%) | |
</details>
Microbench: all 507 measures within bounds.
<details><summary>Hot code per txn</summary>
| path | base | head | change |
|---|---|---|---|
| pooler | 115.5 KB | 115.1 KB | **-0.4 KB** |
| sharding-w2 | 168.2 KB | 168.9 KB | **+0.7 KB** |
pooler:
| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::client::query_engine | 20.4 KB | 20.2 KB | -0.1 KB |
| tokio | 18.6 KB | 18.6 KB | +0.0 KB |
| pgdog::backend::pool | 17.8 KB | 17.7 KB | -0.1 KB |
| aws-lc (TLS crypto) | 10.8 KB | 10.7 KB | -0.1 KB |
| rustls (TLS) | 8.4 KB | 8.4 KB | +0.1 KB |
| pgdog::frontend (other) | 8.1 KB | 7.9 KB | -0.1 KB |
| pgdog::backend::server | 7.9 KB | 7.7 KB | -0.2 KB |
| other | 6.4 KB | 6.4 KB | +0.0 KB |
| everything else | 17.2 KB | 17.4 KB | +0.1 KB |
| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 5.6 KB | -0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 5.4 KB | +0.1 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 4.4 KB | +0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 3.4 KB | -0.1 KB |
| `tokio::runtime::task::raw::poll` | 3.2 KB | -0.1 KB |
sharding-w2:
| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::router | 29.1 KB | 29.1 KB | -0.1 KB |
| pgdog::frontend::client::query_engine | 21.8 KB | 21.6 KB | -0.2 KB |
| tokio | 18.4 KB | 18.4 KB | +0.0 KB |
| pgdog::backend::pool | 18.1 KB | 18.4 KB | +0.2 KB |
| aws-lc (TLS crypto) | 13.0 KB | 13.2 KB | +0.2 KB |
| other | 10.1 KB | 10.4 KB | +0.2 KB |
| pgdog::frontend (other) | 8.8 KB | 8.9 KB | +0.1 KB |
| pgdog::backend::server | 8.0 KB | 8.2 KB | +0.2 KB |
| everything else | 40.8 KB | 40.8 KB | -0.1 KB |
| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 6.3 KB | +0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 5.3 KB | +0.0 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 4.1 KB | -0.1 KB |
| `tokio::runtime::task::raw::poll` | 3.4 KB | +0.2 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 3.3 KB | +0.0 KB |
</details>
<details><summary>lb flamegraph vs 65f51c70</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...).
</details>
<details><summary>microbench flamegraph vs 65f51c70</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...).
</details>
<details><summary>pooler flamegraph vs 65f51c70</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...).
</details>
<details><summary>sharding-w2 flamegraph vs 65f51c70</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...).
</details>
<details><summary>sharding-w8 flamegraph vs 65f51c70</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...).
</details>
<details><summary>Logs</summary>
[lb](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/murex-agg-sharding-key-...), [microbench](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/murex-agg-sharding-key-...), [pooler](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/murex-agg-sharding-key-...), [sharding-w2](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/murex-agg-sharding-key-...), [sharding-w8](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/murex-agg-sharding-key-...), [verdict.json](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/murex-agg-sharding-key-...)
</details>
Compared 7bbe99ad with 65f51c70 on c8g.metal-24xl (on-demand).
^ permalink raw [nested|flat] 5+ messages in thread
* Re: [pgdogdev/pgdog] PR #2072: refactor(cross-shard): sharding key lookup for aggregate rewrite
@ 2026-11-19 14:06 "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
3 siblings, 0 replies; 5+ messages in thread
From: github-actions[bot] (bot) (@github-actions[bot]) @ 2026-11-19 14:06 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
<!-- metal-bench-x86 -->
**Metal bench (x86) passed.** All 520 measures within bounds.
| suite | throughput | cpu/txn | user cycles/txn | user instr/txn | kernel cycles/txn | kernel instr/txn | ctx/txn |
|---|---|---|---|---|---|---|---|
| pooler | | 25.77 (-0.0% ± 0.3%) | 18,544 (+0.1% ± 0.2%) | 32,799 (-0.0% ± 0.04%) | 33,675 (-0.0% ± 0.3%) | 36,514 (-0.2% ± 0.3%) | |
| lb | | 26.18 (+0.1% ± 0.4%) | 19,093 (-0.1% ± 0.09%) | 33,779 (-0.0% ± 0.05%) | 33,808 (+0.1% ± 0.2%) | 37,316 (-0.1% ± 0.3%) | |
| sharding-w2 | | 42.4 (+0.2% ± 7.5%) | 47,936 (+0.4% ± 15.0%) | 98,725 (+0.0% ± 0.1%) | 38,787 (-0.2% ± 1.9%) | 40,061 (-0.0% ± 0.6%) | 1.517 (-0.0% ± 3.2%) |
| sharding-w8 | 206,253 (+0.1% ± 1.2%) | 32.97 (-0.2% ± 1.5%) | 45,983 (-0.4% ± 2.3%) | 53,048 (+0.0% ± 0.1%) | 31,396 (+0.0% ± 1.0%) | 32,506 (+0.0% ± 0.2%) | 0.173 (+0.4% ± 2.2%) |
<details><summary>Counters per txn</summary>
| suite | frontend stall | l1i refill | backend stall |
|---|---|---|---|
| pooler | 3,117 (-0.1% ± 0.6%) | 1,073 (-0.4% ± 0.8%) | 39,920 (+0.1% ± 0.5%) |
| lb | 3,280 (-0.5% ± 0.5%) | 1,110 (-0.4% ± 0.6%) | 40,667 (+0.2% ± 0.4%) |
| sharding-w2 | 8,234 (+23.4% ± 142.4%) | 1,636 (+1.6% ± 17.6%) | 107,641 (+0.8% ± 29.5%) |
| sharding-w8 | 3,355 (+0.4% ± 3.1%) | 890 (+0.5% ± 1.0%) | 178,158 (-0.7% ± 3.2%) |
</details>
Microbench: all 507 measures within bounds.
<details><summary>Microbench latency: 3 benches changed more than 3%</summary>
| benchmark | latency | baseline | change | instructions change |
|---|---|---|---|---|
| simple_cold::order_limit_offset | 8,998 ns | 8,668 ns | +3.80% ± 0.72% | +0.04% ± 0.00% |
| extended_evicting::having | 18,084 ns | 18,702 ns | -3.30% ± 0.72% | -0.00% ± 0.00% |
| extended_cold::having | 18,012 ns | 18,572 ns | -3.01% ± 0.94% | -0.00% ± 0.00% |
</details>
<details><summary>Hot code per txn</summary>
| path | base | head | change |
|---|---|---|---|
| pooler | 117.6 KB | 115.6 KB | **-2.1 KB** |
| sharding-w2 | 174.4 KB | 168.3 KB | **-6.1 KB** |
pooler:
| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::client::query_engine | 23.3 KB | 22.6 KB | -0.8 KB |
| pgdog::backend::pool | 18.6 KB | 18.4 KB | -0.2 KB |
| tokio | 18.5 KB | 18.4 KB | -0.1 KB |
| pgdog::frontend (other) | 9.8 KB | 9.8 KB | +0.0 KB |
| aws-lc (TLS crypto) | 8.6 KB | 8.4 KB | -0.2 KB |
| pgdog::backend::server | 8.1 KB | 7.9 KB | -0.1 KB |
| rustls (TLS) | 7.9 KB | 7.9 KB | -0.1 KB |
| other | 6.2 KB | 5.9 KB | -0.2 KB |
| everything else | 16.6 KB | 16.1 KB | -0.4 KB |
| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 7.4 KB | -0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 6.2 KB | -0.2 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 5.0 KB | -0.2 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 3.9 KB | -0.1 KB |
| `tokio::runtime::task::raw::poll` | 3.6 KB | +0.0 KB |
sharding-w2:
| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::router | 28.7 KB | 28.4 KB | -0.3 KB |
| pgdog::frontend::client::query_engine | 24.9 KB | 23.3 KB | -1.6 KB |
| pgdog::backend::pool | 19.6 KB | 18.9 KB | -0.8 KB |
| tokio | 18.8 KB | 18.7 KB | -0.1 KB |
| pgdog::frontend (other) | 11.1 KB | 10.7 KB | -0.4 KB |
| other | 11.0 KB | 10.2 KB | -0.8 KB |
| aws-lc (TLS crypto) | 10.2 KB | 10.1 KB | -0.1 KB |
| pgdog::backend::server | 9.1 KB | 8.8 KB | -0.3 KB |
| everything else | 41.2 KB | 39.2 KB | -1.9 KB |
| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 8.2 KB | -0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 6.2 KB | -0.1 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 4.5 KB | -0.4 KB |
| `pgdog::frontend::router::parser::query::select::<impl pgdog::frontend::router::parser::que` | 4.2 KB | -0.1 KB |
| `tokio::runtime::task::raw::poll` | 4.0 KB | +0.0 KB |
</details>
<details><summary>lb flamegraph vs 65f51c70</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...).
</details>
<details><summary>microbench flamegraph vs 65f51c70</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...).
</details>
<details><summary>pooler flamegraph vs 65f51c70</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...).
</details>
<details><summary>sharding-w2 flamegraph vs 65f51c70</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...).
</details>
<details><summary>sharding-w8 flamegraph vs 65f51c70</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...).
</details>
<details><summary>Logs</summary>
[lb](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/murex-agg-sharding-...), [microbench](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/murex-agg-sharding-...), [pooler](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/murex-agg-sharding-...), [sharding-w2](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/murex-agg-sharding-...), [sharding-w8](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/murex-agg-sharding-...), [verdict.json](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/murex-agg-sharding-...)
</details>
Compared 7bbe99ad with 65f51c70 on c7i.metal-48xl (on-demand).
^ permalink raw [nested|flat] 5+ messages in thread
* Re: [pgdogdev/pgdog] PR #2072 (approved): refactor(cross-shard): sharding key lookup for aggregate rewrite
@ 2026-11-19 19:48 "levkk (@levkk)" <noreply+levkk@github.com>
3 siblings, 0 replies; 5+ messages in thread
From: levkk (@levkk) @ 2026-11-19 19:48 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
^ permalink raw [nested|flat] 5+ messages in thread
end of thread, other threads:[~2026-11-19 19:48 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-11-19 13:21 [pgdogdev/pgdog] PR #2072: refactor(cross-shard): sharding key lookup for aggregate rewrite "murex971 (@murex971)" <noreply+murex971@github.com>
2026-11-19 13:37 ` "codecov[bot] (bot) (@codecov[bot])" <noreply+codecov[bot]@github.com>
2026-11-19 14:03 ` "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
2026-11-19 14:06 ` "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
2026-11-19 19:48 ` Re: [pgdogdev/pgdog] PR #2072 (approved): refactor(cross-shard): sharding key lookup for aggregate rewrite "levkk (@levkk)" <noreply+levkk@github.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox