pg.ddx.io  pgdogdev/pgdog GitHub issues and pull requests  
help / color / mirror / Atom feed
[pgdogdev/pgdog] PR #2072: refactor(cross-shard): sharding key lookup for aggregate rewrite
5+ messages / 4 participants
[nested] [flat]

* [pgdogdev/pgdog] PR #2072: refactor(cross-shard): sharding key lookup for aggregate rewrite
@ 2026-11-19 13:21 "murex971 (@murex971)" <noreply+murex971@github.com>
  0 siblings, 0 replies; 5+ messages in thread

From: murex971 (@murex971) @ 2026-11-19 13:21 UTC (permalink / raw)
  To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>

### Summary

Groundwork for cross-shard `DISTINCT` aggregates. No behaviour change.

When `DISTINCT` is on the sharding key, or `GROUP BY` includes it, shards
never share a value or a group, so per-shard results can be added up.
The aggregate rewrite needs to know when that is the case.

- Move the router's sharding key lookup to `ShardedTables::sharding_key`
  so the rewrite can reuse it. The router calls the same method.
- Add `ShardingKey`, which answers whether an expression or the
  `GROUP BY` is the sharding key of the table a `SELECT` reads. It only
  answers for a single plain table and a bare column reference; joins,
  subqueries, CTEs and expressions over the key are treated as "no".



inline patch 1: pgdog/src/backend/replication/sharded_tables.rs, pgdog/src/frontend/router/parser/rewrite/statement/aggregate/mod.rs, pgdog/src/frontend/router/parser/rewrite/statement/aggregate/sharding_key.rs and 1 more (+262/-26 lines) download inline-1.patchdiff --git a/pgdog/src/backend/replication/sharded_tables.rs b/pgdog/src/backend/replication/sharded_tables.rs index b8cbb358f..46b387e61 100644 --- a/pgdog/src/backend/replication/sharded_tables.rs +++ b/pgdog/src/backend/replication/sharded_tables.rs @@ -247,6 +247,32 @@ impl ShardedTables { None } + + /// Sharded table config for a column, if the column is a sharding key. + /// + /// Configs that name a table match that table's column. Configs with only + /// a column match that column on any table. + pub(crate) fn sharding_key(&self, column: Column<'_>) -> Option<&ShardedTable> { + // Omnisharded config takes priority over sharded tables: + // a sharding key on an omnisharded table doesn't route. + if column + .table + .is_some_and(|name| self.omnishards().contains_key(name)) + { + return None; + } + + if let Some(table) = self.get_table(column) + && table.name.is_some() + { + return Some(table); + } + + // Column-only config: user explicitly wants any table with this column to be sharded + self.tables() + .iter() + .find(|table| table.name.is_none() && table.column == column.name) + } } #[cfg(test)] diff --git a/pgdog/src/frontend/router/parser/rewrite/statement/aggregate/mod.rs b/pgdog/src/frontend/router/parser/rewrite/statement/aggregate/mod.rs index 5709b4589..119a8f468 100644 --- a/pgdog/src/frontend/router/parser/rewrite/statement/aggregate/mod.rs +++ b/pgdog/src/frontend/router/parser/rewrite/statement/aggregate/mod.rs @@ -1,4 +1,7 @@ mod engine; +// Used by the DISTINCT aggregate rewrite. +#[cfg_attr(not(test), expect(dead_code))] +mod sharding_key; pub(crate) use super::projection::AggregateHelper; diff --git a/pgdog/src/frontend/router/parser/rewrite/statement/aggregate/sharding_key.rs b/pgdog/src/frontend/router/parser/rewrite/statement/aggregate/sharding_key.rs new file mode 100644 index 000000000..83f358d52 --- /dev/null +++ b/pgdog/src/frontend/router/parser/rewrite/statement/aggregate/sharding_key.rs @@ -0,0 +1,228 @@ +//! Sharding key of the table a SELECT reads. +//! +//! A sharding key value lives on one shard, so shards never share one. Each +//! shard's `DISTINCT` aggregate over the key covers values no other shard +//! has, and a group that includes the key is complete on its shard. + +use itertools::Itertools; +use pg_raw_parse::{Node, nodes}; + +use crate::backend::replication::ShardedTables; +use crate::frontend::router::parser::{Column, Table}; + +/// Answers whether an expression is the sharding key of the queried table. +pub(crate) struct ShardingKey<'a> { + table: Table<'a>, + tables: &'a ShardedTables, +} + +impl<'a> ShardingKey<'a> { + /// `None` unless the SELECT reads exactly one plain table. Joins, + /// subqueries and functions in `FROM` can repeat a key on several shards, + /// and a CTE can shadow the table's name. + pub(crate) fn new(select: &'a nodes::SelectStmt, tables: &'a ShardedTables) -> Option<Self> { + if select.with_clause().is_some() { + return None; + } + let Ok(Node::RangeVar(range_var)) = select.from_clause().iter().exactly_one() else { + return None; + }; + + Some(Self { + table: Table::from(range_var), + tables, + }) + } + + /// Whether `node` is a bare reference to the sharding key column. + /// Expressions over the key, e.g. `user_id % 2`, don't count: + /// two shards can produce the same result. + pub(crate) fn matches(&self, node: Node<'_>) -> bool { + let Node::ColumnRef(column) = node else { + return false; + }; + let Ok(mut column) = Column::try_from(column) else { + return false; + }; + if !self.qualifier_matches(&column) { + return false; + } + column.table = Some(self.table.name); + column.schema = self.table.schema; + + self.tables.sharding_key(column).is_some() + } + + /// Whether `GROUP BY` includes the sharding key, by name or by SELECT + /// list position, so that no group spans shards. + pub(crate) fn in_group_by(&self, select: &nodes::SelectStmt) -> bool { + select.group_clause().iter().any(|item| match item { + // Postgres counts SELECT list positions from 1. + Node::A_Const(ordinal) => ordinal + .val() + .and_then(|value| value.numeric_value::<i32>()) + .and_then(|ordinal| usize::try_from(ordinal).ok()?.checked_sub(1)) + .and_then(|position| select.target_list().iter().nth(position)) + .is_some_and(|target| self.matches(target.val())), + item => self.matches(item), + }) + } + + /// An aliased table is only known by its alias. + fn qualifier_matches(&self, column: &Column<'_>) -> bool { + match (column.schema, column.table, self.table.alias) { + (None, None, _) => true, + (None, Some(qualifier), Some(alias)) => qualifier == alias, + (None, Some(qualifier), None) => qualifier == self.table.name, + (Some(schema), Some(qualifier), None) => { + qualifier == self.table.name && self.table.schema == Some(schema) + } + _ => false, + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::frontend::router::sharding::ShardedTable; + use pg_raw_parse::{Owned, make}; + use pgdog_config::{OmnishardedTable, SystemCatalogsBehavior}; + + fn select(sql: &str) -> Owned<nodes::SelectStmt> { + match pg_raw_parse::parse(sql) + .expect("valid SQL") + .stmts() + .next() + .expect("one statement") + { + Node::SelectStmt(stmt) => make::owned(|mem| mem.make_unique(stmt)), + _ => panic!("not a select"), + } + } + + fn config(name: Option<&str>, column: &str) -> ShardedTable { + ShardedTable { + name: name.map(Into::into), + column: column.into(), + ..Default::default() + } + } + + fn orders_by_user() -> ShardedTables { + ShardedTables::from([config(Some("orders"), "user_id")].as_slice()) + } + + /// Whether the first SELECT list item and the GROUP BY are on the sharding key. + fn check(sql: &str, tables: &ShardedTables) -> Option<(bool, bool)> { + let select = select(sql); + let key = ShardingKey::new(&select, tables)?; + let first = select.target_list().iter().next().expect("one target"); + + Some((key.matches(first.val()), key.in_group_by(&select))) + } + + #[test] + fn matches_bare_key_column() { + let tables = orders_by_user(); + + for (sql, expected) in [ + ("SELECT user_id FROM orders", true), + ("SELECT orders.user_id FROM orders", true), + ("SELECT public.orders.user_id FROM public.orders", true), + ("SELECT o.user_id FROM orders o", true), + ("SELECT product_id FROM orders", false), + // Two shards can produce the same value. + ("SELECT user_id % 2 FROM orders", false), + ("SELECT user_id::text FROM orders", false), + // The alias hides the table name. + ("SELECT orders.user_id FROM orders o", false), + ("SELECT other.user_id FROM orders", false), + ("SELECT other.orders.user_id FROM public.orders", false), + ("SELECT user_id FROM users", false), + ] { + assert_eq!( + check(sql, &tables).map(|(first, _)| first), + Some(expected), + "{sql}" + ); + } + } + + #[test] + fn requires_one_plain_table() { + let tables = orders_by_user(); + + for sql in [ + "SELECT user_id FROM orders JOIN users ON users.id = orders.user_id", + "SELECT user_id FROM orders, users", + "SELECT user_id FROM (SELECT user_id FROM orders) AS orders", + "SELECT user_id FROM generate_series(1, 3) AS orders(user_id)", + "WITH orders AS (SELECT 1 AS user_id) SELECT user_id FROM orders", + "SELECT user_id FROM orders UNION SELECT user_id FROM orders", + "SELECT 1", + ] { + assert!(check(sql, &tables).is_none(), "{sql}"); + } + } + + #[test] + fn finds_key_in_group_by() { + let tables = orders_by_user(); + + for (sql, expected) in [ + ("SELECT count(*) FROM orders GROUP BY user_id", true), + ( + "SELECT count(*) FROM orders GROUP BY status, o.user_id", + false, + ), + ( + "SELECT count(*) FROM orders o GROUP BY status, o.user_id", + true, + ), + ( + "SELECT status, user_id, count(*) FROM orders GROUP BY 1, 2", + true, + ), + ("SELECT status, count(*) FROM orders GROUP BY 1", false), + ("SELECT count(*) FROM orders GROUP BY status", false), + ("SELECT count(*) FROM orders GROUP BY user_id % 2", false), + ("SELECT count(*) FROM orders", false), + // The grand total group spans shards. + ( + "SELECT count(*) FROM orders GROUP BY ROLLUP (user_id)", + false, + ), + ] { + assert_eq!( + check(sql, &tables).map(|(_, group)| group), + Some(expected), + "{sql}" + ); + } + } + + #[test] + fn follows_sharded_table_config() { + // A config without a table name covers the column on every table. + let column_only = ShardedTables::from([config(None, "user_id")].as_slice()); + assert_eq!( + check("SELECT user_id FROM anything", &column_only), + Some((true, false)) + ); + + let omnisharded = ShardedTables::new( + vec![config(None, "user_id")], + vec![OmnishardedTable { + name: "settings".into(), + sticky_routing: false, + }], + false, + SystemCatalogsBehavior::default(), + ); + assert_eq!( + check("SELECT user_id FROM settings", &omnisharded), + Some((false, false)) + ); + } +} diff --git a/pgdog/src/frontend/router/parser/statement.rs b/pgdog/src/frontend/router/parser/statement.rs index 74fe69df8..e810b0062 100644 --- a/pgdog/src/frontend/router/parser/statement.rs +++ b/pgdog/src/frontend/router/parser/statement.rs @@ -660,32 +660,11 @@ impl<'a, 'b: 'a> StatementParser<'a, 'b> { table_name: Option<&str>, schema: Option<&str>, ) -> Option<&'b ShardedTable> { - // Omnisharded config takes priority over sharded tables: - // a sharding key on an omnisharded table doesn't route. - if table_name.is_some_and(|name| self.schema.tables.omnishards().contains_key(name)) { - return None; - } - - // Try named table configs first - if let Some(table_name) = table_name { - let column = Column { - name: column_name, - table: Some(table_name), - schema, - }; - if let Some(sharded_table) = self.schema.tables().get_table(column) - && sharded_table.name.is_some() - { - return Some(sharded_table); - } - } - - // Column-only config: user explicitly wants any table with this column to be sharded - self.schema - .tables - .tables() - .iter() - .find(|t| t.name.is_none() && t.column == column_name) + self.schema.tables().sharding_key(Column { + name: column_name, + table: table_name, + schema, + }) } fn compute_shard(
^ permalink raw [nested|flat] 5+ messages in thread

* Re: [pgdogdev/pgdog] PR #2072: refactor(cross-shard): sharding key lookup for aggregate rewrite
@ 2026-11-19 13:37 ` "codecov[bot] (bot) (@codecov[bot])" <noreply+codecov[bot]@github.com>
  3 siblings, 0 replies; 5+ messages in thread

From: codecov[bot] (bot) (@codecov[bot]) @ 2026-11-19 13:37 UTC (permalink / raw)
  To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>

## [Codecov](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2072?dropdown=coverage&src=pr&el=h1&utm_me...) Report
:x: Patch coverage is `98.18182%` with `3 lines` in your changes missing coverage. Please review.
| [Files with missing lines](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2072?dropdown=coverage&src=pr&el=tree&utm_...) | Patch % | Lines |
|---|---|---|
| [...parser/rewrite/statement/aggregate/sharding\_key.rs](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2072?src=pr&el=tree&utm_medium=referral&ut...) | 97.94% | [3 Missing :warning: ](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2072?src=pr&el=tree&utm_medium=referral&ut...) |

:loudspeaker: Thoughts on this report? [Let us know!](https://github.com/codecov/feedback/issues/255)

^ permalink  raw  [nested|flat] 5+ messages in thread

* Re: [pgdogdev/pgdog] PR #2072: refactor(cross-shard): sharding key lookup for aggregate rewrite
@ 2026-11-19 14:03 ` "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
  3 siblings, 0 replies; 5+ messages in thread

From: github-actions[bot] (bot) (@github-actions[bot]) @ 2026-11-19 14:03 UTC (permalink / raw)
  To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>

<!-- metal-bench -->

**Metal bench (arm64) passed.** All 520 measures within bounds.

| suite | throughput | cpu/txn | user cycles/txn | user instr/txn | kernel cycles/txn | kernel instr/txn | ctx/txn |
|---|---|---|---|---|---|---|---|
| pooler |  | 19.03 (+0.3% ± 1.2%) | 18,068 (-0.0% ± 0.8%) | 31,448 (-0.0% ± 0.04%) | 30,642 (+0.6% ± 2.1%) | 51,112 (+0.0% ± 0.4%) |  |
| lb |  | 19.59 (+0.5% ± 1.6%) | 18,653 (+0.2% ± 1.0%) | 32,348 (-0.0% ± 0.05%) | 31,300 (+1.1% ± 2.3%) | 52,228 (+0.2% ± 0.5%) |  |
| sharding-w2 |  | 32.28 (-0.2% ± 1.7%) | 43,386 (-0.2% ± 0.9%) | 94,901 (+0.0% ± 0.02%) | 40,460 (-0.2% ± 3.0%) | 53,742 (+0.1% ± 0.1%) | 1.547 (+0.1% ± 0.1%) |
| sharding-w8 | 243,532 (+0.5% ± 3.5%) | 29.43 (-0.3% ± 3.3%) | 44,253 (+0.1% ± 3.0%) | 50,494 (-0.0% ± 0.09%) | 37,021 (-0.8% ± 3.7%) | 42,373 (-0.0% ± 0.09%) | 0.166 (-1.0% ± 0.8%) |

<details><summary>Counters per txn</summary>

| suite | frontend stall | l1i refill | backend stall |
|---|---|---|---|
| pooler | 7,789 (+0.0% ± 2.1%) | 2,677 (-0.1% ± 0.9%) |  |
| lb | 8,142 (+0.7% ± 2.4%) | 2,728 (+0.2% ± 1.1%) |  |
| sharding-w2 | 17,981 (-0.3% ± 1.8%) | 4,167 (-0.2% ± 0.9%) |  |
| sharding-w8 | 20,411 (+1.0% ± 5.0%) | 4,184 (+0.4% ± 2.5%) |  |
</details>

Microbench: all 507 measures within bounds.

<details><summary>Hot code per txn</summary>

| path | base | head | change |
|---|---|---|---|
| pooler | 115.5 KB | 115.1 KB | **-0.4 KB** |
| sharding-w2 | 168.2 KB | 168.9 KB | **+0.7 KB** |

pooler:

| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::client::query_engine | 20.4 KB | 20.2 KB | -0.1 KB |
| tokio | 18.6 KB | 18.6 KB | +0.0 KB |
| pgdog::backend::pool | 17.8 KB | 17.7 KB | -0.1 KB |
| aws-lc (TLS crypto) | 10.8 KB | 10.7 KB | -0.1 KB |
| rustls (TLS) | 8.4 KB | 8.4 KB | +0.1 KB |
| pgdog::frontend (other) | 8.1 KB | 7.9 KB | -0.1 KB |
| pgdog::backend::server | 7.9 KB | 7.7 KB | -0.2 KB |
| other | 6.4 KB | 6.4 KB | +0.0 KB |
| everything else | 17.2 KB | 17.4 KB | +0.1 KB |

| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 5.6 KB | -0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 5.4 KB | +0.1 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 4.4 KB | +0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 3.4 KB | -0.1 KB |
| `tokio::runtime::task::raw::poll` | 3.2 KB | -0.1 KB |

sharding-w2:

| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::router | 29.1 KB | 29.1 KB | -0.1 KB |
| pgdog::frontend::client::query_engine | 21.8 KB | 21.6 KB | -0.2 KB |
| tokio | 18.4 KB | 18.4 KB | +0.0 KB |
| pgdog::backend::pool | 18.1 KB | 18.4 KB | +0.2 KB |
| aws-lc (TLS crypto) | 13.0 KB | 13.2 KB | +0.2 KB |
| other | 10.1 KB | 10.4 KB | +0.2 KB |
| pgdog::frontend (other) | 8.8 KB | 8.9 KB | +0.1 KB |
| pgdog::backend::server | 8.0 KB | 8.2 KB | +0.2 KB |
| everything else | 40.8 KB | 40.8 KB | -0.1 KB |

| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 6.3 KB | +0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 5.3 KB | +0.0 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 4.1 KB | -0.1 KB |
| `tokio::runtime::task::raw::poll` | 3.4 KB | +0.2 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 3.3 KB | +0.0 KB |

</details>

<details><summary>lb flamegraph vs 65f51c70</summary>

![lb](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...) for hover, zoom and search.

![lb removed](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...).
</details>
<details><summary>microbench flamegraph vs 65f51c70</summary>

![microbench](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...) for hover, zoom and search.

![microbench removed](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...).
</details>
<details><summary>pooler flamegraph vs 65f51c70</summary>

![pooler](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...) for hover, zoom and search.

![pooler removed](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...).
</details>
<details><summary>sharding-w2 flamegraph vs 65f51c70</summary>

![sharding-w2](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...) for hover, zoom and search.

![sharding-w2 removed](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...).
</details>
<details><summary>sharding-w8 flamegraph vs 65f51c70</summary>

![sharding-w8](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...) for hover, zoom and search.

![sharding-w8 removed](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/murex-agg-sharding-key-lookup/7bbe99ad12b066745db...).
</details>

<details><summary>Logs</summary>

[lb](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/murex-agg-sharding-key-...), [microbench](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/murex-agg-sharding-key-...), [pooler](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/murex-agg-sharding-key-...), [sharding-w2](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/murex-agg-sharding-key-...), [sharding-w8](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/murex-agg-sharding-key-...), [verdict.json](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/murex-agg-sharding-key-...)
</details>

Compared 7bbe99ad with 65f51c70 on c8g.metal-24xl (on-demand).


^ permalink  raw  [nested|flat] 5+ messages in thread

* Re: [pgdogdev/pgdog] PR #2072: refactor(cross-shard): sharding key lookup for aggregate rewrite
@ 2026-11-19 14:06 ` "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
  3 siblings, 0 replies; 5+ messages in thread

From: github-actions[bot] (bot) (@github-actions[bot]) @ 2026-11-19 14:06 UTC (permalink / raw)
  To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>

<!-- metal-bench-x86 -->

**Metal bench (x86) passed.** All 520 measures within bounds.

| suite | throughput | cpu/txn | user cycles/txn | user instr/txn | kernel cycles/txn | kernel instr/txn | ctx/txn |
|---|---|---|---|---|---|---|---|
| pooler |  | 25.77 (-0.0% ± 0.3%) | 18,544 (+0.1% ± 0.2%) | 32,799 (-0.0% ± 0.04%) | 33,675 (-0.0% ± 0.3%) | 36,514 (-0.2% ± 0.3%) |  |
| lb |  | 26.18 (+0.1% ± 0.4%) | 19,093 (-0.1% ± 0.09%) | 33,779 (-0.0% ± 0.05%) | 33,808 (+0.1% ± 0.2%) | 37,316 (-0.1% ± 0.3%) |  |
| sharding-w2 |  | 42.4 (+0.2% ± 7.5%) | 47,936 (+0.4% ± 15.0%) | 98,725 (+0.0% ± 0.1%) | 38,787 (-0.2% ± 1.9%) | 40,061 (-0.0% ± 0.6%) | 1.517 (-0.0% ± 3.2%) |
| sharding-w8 | 206,253 (+0.1% ± 1.2%) | 32.97 (-0.2% ± 1.5%) | 45,983 (-0.4% ± 2.3%) | 53,048 (+0.0% ± 0.1%) | 31,396 (+0.0% ± 1.0%) | 32,506 (+0.0% ± 0.2%) | 0.173 (+0.4% ± 2.2%) |

<details><summary>Counters per txn</summary>

| suite | frontend stall | l1i refill | backend stall |
|---|---|---|---|
| pooler | 3,117 (-0.1% ± 0.6%) | 1,073 (-0.4% ± 0.8%) | 39,920 (+0.1% ± 0.5%) |
| lb | 3,280 (-0.5% ± 0.5%) | 1,110 (-0.4% ± 0.6%) | 40,667 (+0.2% ± 0.4%) |
| sharding-w2 | 8,234 (+23.4% ± 142.4%) | 1,636 (+1.6% ± 17.6%) | 107,641 (+0.8% ± 29.5%) |
| sharding-w8 | 3,355 (+0.4% ± 3.1%) | 890 (+0.5% ± 1.0%) | 178,158 (-0.7% ± 3.2%) |
</details>

Microbench: all 507 measures within bounds.

<details><summary>Microbench latency: 3 benches changed more than 3%</summary>

| benchmark | latency | baseline | change | instructions change |
|---|---|---|---|---|
| simple_cold::order_limit_offset | 8,998 ns | 8,668 ns | +3.80% ± 0.72% | +0.04% ± 0.00% |
| extended_evicting::having | 18,084 ns | 18,702 ns | -3.30% ± 0.72% | -0.00% ± 0.00% |
| extended_cold::having | 18,012 ns | 18,572 ns | -3.01% ± 0.94% | -0.00% ± 0.00% |
</details>

<details><summary>Hot code per txn</summary>

| path | base | head | change |
|---|---|---|---|
| pooler | 117.6 KB | 115.6 KB | **-2.1 KB** |
| sharding-w2 | 174.4 KB | 168.3 KB | **-6.1 KB** |

pooler:

| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::client::query_engine | 23.3 KB | 22.6 KB | -0.8 KB |
| pgdog::backend::pool | 18.6 KB | 18.4 KB | -0.2 KB |
| tokio | 18.5 KB | 18.4 KB | -0.1 KB |
| pgdog::frontend (other) | 9.8 KB | 9.8 KB | +0.0 KB |
| aws-lc (TLS crypto) | 8.6 KB | 8.4 KB | -0.2 KB |
| pgdog::backend::server | 8.1 KB | 7.9 KB | -0.1 KB |
| rustls (TLS) | 7.9 KB | 7.9 KB | -0.1 KB |
| other | 6.2 KB | 5.9 KB | -0.2 KB |
| everything else | 16.6 KB | 16.1 KB | -0.4 KB |

| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 7.4 KB | -0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 6.2 KB | -0.2 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 5.0 KB | -0.2 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 3.9 KB | -0.1 KB |
| `tokio::runtime::task::raw::poll` | 3.6 KB | +0.0 KB |

sharding-w2:

| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::router | 28.7 KB | 28.4 KB | -0.3 KB |
| pgdog::frontend::client::query_engine | 24.9 KB | 23.3 KB | -1.6 KB |
| pgdog::backend::pool | 19.6 KB | 18.9 KB | -0.8 KB |
| tokio | 18.8 KB | 18.7 KB | -0.1 KB |
| pgdog::frontend (other) | 11.1 KB | 10.7 KB | -0.4 KB |
| other | 11.0 KB | 10.2 KB | -0.8 KB |
| aws-lc (TLS crypto) | 10.2 KB | 10.1 KB | -0.1 KB |
| pgdog::backend::server | 9.1 KB | 8.8 KB | -0.3 KB |
| everything else | 41.2 KB | 39.2 KB | -1.9 KB |

| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 8.2 KB | -0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 6.2 KB | -0.1 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 4.5 KB | -0.4 KB |
| `pgdog::frontend::router::parser::query::select::<impl pgdog::frontend::router::parser::que` | 4.2 KB | -0.1 KB |
| `tokio::runtime::task::raw::poll` | 4.0 KB | +0.0 KB |

</details>

<details><summary>lb flamegraph vs 65f51c70</summary>

![lb](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...) for hover, zoom and search.

![lb removed](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...).
</details>
<details><summary>microbench flamegraph vs 65f51c70</summary>

![microbench](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...) for hover, zoom and search.

![microbench removed](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...).
</details>
<details><summary>pooler flamegraph vs 65f51c70</summary>

![pooler](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...) for hover, zoom and search.

![pooler removed](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...).
</details>
<details><summary>sharding-w2 flamegraph vs 65f51c70</summary>

![sharding-w2](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...) for hover, zoom and search.

![sharding-w2 removed](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...).
</details>
<details><summary>sharding-w8 flamegraph vs 65f51c70</summary>

![sharding-w8](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...) for hover, zoom and search.

![sharding-w8 removed](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...)

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/murex-agg-sharding-key-lookup/7bbe99ad12b0667...).
</details>

<details><summary>Logs</summary>

[lb](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/murex-agg-sharding-...), [microbench](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/murex-agg-sharding-...), [pooler](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/murex-agg-sharding-...), [sharding-w2](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/murex-agg-sharding-...), [sharding-w8](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/murex-agg-sharding-...), [verdict.json](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/murex-agg-sharding-...)
</details>

Compared 7bbe99ad with 65f51c70 on c7i.metal-48xl (on-demand).


^ permalink  raw  [nested|flat] 5+ messages in thread

* Re: [pgdogdev/pgdog] PR #2072 (approved): refactor(cross-shard): sharding key lookup for aggregate rewrite
@ 2026-11-19 19:48 ` "levkk (@levkk)" <noreply+levkk@github.com>
  3 siblings, 0 replies; 5+ messages in thread

From: levkk (@levkk) @ 2026-11-19 19:48 UTC (permalink / raw)
  To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>



^ permalink  raw  [nested|flat] 5+ messages in thread


end of thread, other threads:[~2026-11-19 19:48 UTC | newest]

Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-11-19 13:21 [pgdogdev/pgdog] PR #2072: refactor(cross-shard): sharding key lookup for aggregate rewrite "murex971 (@murex971)" <noreply+murex971@github.com>
2026-11-19 13:37 ` "codecov[bot] (bot) (@codecov[bot])" <noreply+codecov[bot]@github.com>
2026-11-19 14:03 ` "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
2026-11-19 14:06 ` "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
2026-11-19 19:48 ` Re: [pgdogdev/pgdog] PR #2072 (approved): refactor(cross-shard): sharding key lookup for aggregate rewrite "levkk (@levkk)" <noreply+levkk@github.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox