pg.ddx.io pgdogdev/pgdog GitHub issues and pull requests
help / color / mirror / Atom feed[pgdogdev/pgdog] PR #2064: fix: don't route by cols belonging to CTEs / derived tables if same name as sharded table & key
6+ messages / 3 participants
[nested] [flat]
* [pgdogdev/pgdog] PR #2064: fix: don't route by cols belonging to CTEs / derived tables if same name as sharded table & key
@ 2026-11-18 18:24 "jkaczman (@jkaczman)" <noreply+jkaczman@github.com>
0 siblings, 0 replies; 6+ messages in thread
From: jkaczman (@jkaczman) @ 2026-11-18 18:24 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
Previously, if a derived table or CTE had the same exact name of some sharded table, and the derived table / CTE had a column with the same name as the column used for the sharding key represented by that corresponding sharded table, those derived tables / CTEs would be treated as sharded in the same way as the table.
E.g., if we had users sharded by id, the router sent `SELECT * FROM (SELECT 1 AS id) users WHERE users.id = 1` to the shard corresponding to `1` (only), although users there is the derived table (not the actual table users)
Now, the router records what the names in `FROM` refer to (tables, derived tables or CTEs), and thus, now knows only to route by sharding keys which actually belong to real tables.
This is also prep for cross-shard JOINs (#94). To rewrite a JOIN, we'll fetch the rows belonging to the table after `JOIN` from all shards, then replace that table with a derived table with those rows. The derived table maintains the same name of the table that it fetches from to ensure that the rest of the statement still works. E.g., `.. FROM notifications JOIN accounts ON .. WHERE accounts.id = 5` becomes `.. FROM notifications JOIN (SELECT .. FROM unnest(..)) accounts ON .. WHERE accounts.id = 5`.
Without this fix, the router would read `accounts.id = 5` in that statement as the sharding key belonging to the table `accounts`, and send the statement to the shard corresponding to `5` only. However, the rows belonging to `notifications` that the statement joins might be on a variety of shards, maybe even all of them
inline patch 1: pgdog/src/frontend/router/parser/statement.rs (+119/-63 lines) download inline-1.patch
diff --git a/pgdog/src/frontend/router/parser/statement.rs b/pgdog/src/frontend/router/parser/statement.rs
index 74fe69df8..99db592d0 100644
--- a/pgdog/src/frontend/router/parser/statement.rs
+++ b/pgdog/src/frontend/router/parser/statement.rs
@@ -58,11 +58,22 @@ use pgdog_postgres_types::{Array, Datum};
use super::rewrite::statement::insert::array_param_of_select_from_unnest;
+/// The kind of thing that a name in FROM refers to, i.e., a table, a derived table or a CTE
+/// E.g., in WITH c AS (..) SELECT .. FROM users u, (SELECT ..) s, c, the name u refers to the table users (Table), the name s refers to a derived table (Derived), and the name c refers to a CTE (Cte)
+#[derive(Debug, Clone, Copy)]
+enum Relation<'a> {
+ Table(Table<'a>),
+ Derived,
+ Cte,
+}
+
/// Context for searching a SELECT statement, tracking table aliases.
#[derive(Debug, Default, Clone)]
struct SearchContext<'a> {
- /// Maps alias -> full Table (including schema)
- aliases: HashMap<&'a str, Table<'a>>,
+ /// Stores `Relations` encountered
+ /// E.g., for FROM users u JOIN orders ON .., we have u -> Table(users) and orders -> Table(orders).
+ /// Key is alias (u) or table name for tables that don't have alias (orders)
+ relations: HashMap<&'a str, Relation<'a>>,
/// The primary table from the FROM clause (if simple)
table: Option<Table<'a>>,
/// Base tables in this FROM clause, without descending into subqueries.
@@ -73,7 +84,7 @@ struct SearchContext<'a> {
impl<'a> SearchContext<'a> {
/// Build context from a FROM clause, extracting table aliases.
- fn from_from_clause(nodes: &'a list::NodeList) -> Self {
+ fn from_from_clause(nodes: &'a list::NodeList, cte_names: &HashSet<&'a str>) -> Self {
let table = nodes
.into_iter()
.exactly_one()
@@ -85,7 +96,7 @@ impl<'a> SearchContext<'a> {
..Default::default()
};
for node in nodes {
- ctx.extract_table_from_node(node);
+ ctx.extract_table_from_node(node, cte_names);
ctx.extract_joined_columns(node);
}
ctx
@@ -160,33 +171,33 @@ impl<'a> SearchContext<'a> {
});
}
- fn extract_table_from_node(&mut self, node: Node<'a>) {
+ fn extract_table_from_node(&mut self, node: Node<'a>, cte_names: &HashSet<&'a str>) {
match node {
Node::RangeVar(rv) => {
let table = Table::from(rv);
- self.tables.push(table);
- if let Some(alias) = rv.alias() {
- self.aliases
- .insert(alias.aliasname().expect("alias name always present"), table);
- }
+ // Why check `cte_names`? Because CTEs can have the same name as a table
+ // E.g., if we consider the CTE WITH users AS (..) SELECT .. FROM users)
+ // - for FROM users, Postgres then uses the CTE users (instead of the table users)
+ // Names with a schema (e.g., public.users) can't be CTEs (so those are always tables)
+ let relation = if table.schema.is_none() && cte_names.contains(table.name) {
+ Relation::Cte
+ } else {
+ self.tables.push(table);
+ Relation::Table(table)
+ };
+ self.relations
+ .insert(table.alias.unwrap_or(table.name), relation);
}
Node::JoinExpr(join) => {
- self.extract_table_from_node(join.larg());
- self.extract_table_from_node(join.rarg());
+ self.extract_table_from_node(join.larg(), cte_names);
+ self.extract_table_from_node(join.rarg(), cte_names);
}
Node::RangeSubselect(subselect) if let Some(alias) = subselect.alias() => {
- // For subselects, we don't have a real table name
- // but we record the alias anyway for future use
- let aliasname = alias.aliasname().expect("alias name always present");
- self.aliases.insert(
- aliasname,
- Table {
- name: aliasname,
- schema: None,
- alias: None,
- },
+ self.relations.insert(
+ alias.aliasname().expect("alias name always present"),
+ Relation::Derived,
);
}
@@ -194,23 +205,22 @@ impl<'a> SearchContext<'a> {
}
}
- /// Resolve a table reference (which may be an alias) to the actual Table.
- fn resolve_table(&self, name: &str) -> Option<Table<'a>> {
- self.aliases.get(name).copied()
- }
-
- /// Qualify a column with the actual table its table alias refers to.
- fn resolve_column(&self, column: Column<'a>) -> Column<'a> {
- match column
- .table()
- .and_then(|table| self.resolve_table(table.name))
- {
- Some(resolved) => Column {
+ /// Replaces the alias in `column` with the name (and schema) of the table that the alias stands for, e.g., u.id in FROM users u becomes users.id
+ /// We need the table name to check whether the column is a sharding key since the sharding config lists table names (users) not aliases (u)
+ /// Returns `column` as-is if the column doesn't have a table (e.g., id in .. FROM users JOIN orders ON .. WHERE id = 1, where id can be from either table) or if the table that the column belongs to isn't in FROM
+ /// Returns None for columns belonging to derived tables and CTEs (e.g., s.id in FROM (SELECT ..) s), since those don't have any sharding keys
+ fn resolve_column(&self, column: Column<'a>) -> Option<Column<'a>> {
+ let Some(name) = column.table else {
+ return Some(column);
+ };
+ match self.relations.get(name) {
+ Some(Relation::Table(table)) => Some(Column {
name: column.name,
- table: Some(resolved.name),
- schema: resolved.schema,
- },
- None => column,
+ table: Some(table.name),
+ schema: table.schema,
+ }),
+ Some(Relation::Derived | Relation::Cte) => None,
+ None => Some(column),
}
}
}
@@ -620,8 +630,10 @@ impl<'a, 'b: 'a> StatementParser<'a, 'b> {
let table = Table::from(range_var);
ctx.table = Some(table);
if let Some(alias) = range_var.alias() {
- ctx.aliases
- .insert(alias.aliasname().expect("Alias name always present"), table);
+ ctx.relations.insert(
+ alias.aliasname().expect("Alias name always present"),
+ Relation::Table(table),
+ );
}
}
ctx
@@ -833,8 +845,10 @@ impl<'a, 'b: 'a> StatementParser<'a, 'b> {
}
fn search_stmt(&mut self, stmt: Node<'a>) -> ControlFlow<Result<Shard, Error>> {
- let mut ctx = match stmt {
- Node::SelectStmt(s) => SearchContext::from_from_clause(s.from_clause()),
+ let ctx = match stmt {
+ Node::SelectStmt(s) => {
+ SearchContext::from_from_clause(s.from_clause(), &self.walk().cte_names)
+ }
Node::UpdateStmt(s) => self.context_from_relation(s.relation()),
Node::DeleteStmt(s) => self.context_from_relation(s.relation()),
Node::InsertStmt(s) => {
@@ -847,11 +861,6 @@ impl<'a, 'b: 'a> StatementParser<'a, 'b> {
_ => return ControlFlow::Continue(()),
};
- // A CTE can shadow a configured table without preserving its columns.
- let cte_names = &self.walk().cte_names;
- ctx.tables
- .retain(|table| table.schema.is_some() || !cte_names.contains(table.name));
-
if let Node::SelectStmt(select) = stmt
&& !ctx.joined_columns.is_empty()
{
@@ -913,16 +922,22 @@ impl<'a, 'b: 'a> StatementParser<'a, 'b> {
// For ANY expressions with sharding columns, we can't reliably
// parse array literals or parameters, so route to all shards.
(SearchResult::Column(column), _, true)
- if self.get_sharded_table(ctx.resolve_column(column)).is_some() =>
+ if ctx
+ .resolve_column(column)
+ .and_then(|column| self.get_sharded_table(column))
+ .is_some() =>
{
ControlFlow::Break(Ok(Shard::All))
}
(SearchResult::Column(column), Some(values), false)
| (values, Some(SearchResult::Column(column)), false) => {
+ let Some(column) = ctx.resolve_column(column) else {
+ return Recurse::no();
+ };
let shards = values
.iter()
.filter_map(|value| {
- self.compute_shard_with_ctx(column, value.clone(), ctx)
+ self.compute_shard_of_resolved_column(column, value.clone())
.transpose()
})
.collect::<Result<Vec<_>, _>>()
@@ -956,8 +971,9 @@ impl<'a, 'b: 'a> StatementParser<'a, 'b> {
// The array doesn't have to be the same type as the key, e.g., id could be a bigint while $1 is a numeric array such as {1.5}.
// Postgres is fine with comparing a bigint to a numeric, however, if we tried to compute the shard of 1.5 for a bigint key, PgDog would return an error to the client.
// Therefore, we skip this IN, and the statement is routed by whatever else is in the WHERE (which is also what we do for id IN (1.5)).
- let sharding_key_data_type = self
- .get_sharded_table(ctx.resolve_column(sharding_key_column))
+ let sharding_key_data_type = ctx
+ .resolve_column(sharding_key_column)
+ .and_then(|column| self.get_sharded_table(column))
.map(|sharded_table| sharded_table.data_type);
if sharding_key_data_type.is_some_and(|sharding_key_data_type| {
@@ -969,13 +985,15 @@ impl<'a, 'b: 'a> StatementParser<'a, 'b> {
return Recurse::no();
}
+ let Some(sharding_key_column) = ctx.resolve_column(sharding_key_column) else {
+ return Recurse::no();
+ };
let shards_of_array_elements = array_elements_as_text
.iter()
.filter_map(|array_element_as_text| {
- self.compute_shard_with_ctx(
+ self.compute_shard_of_resolved_column(
sharding_key_column,
Value::String(array_element_as_text),
- ctx,
)
.transpose()
})
@@ -1060,7 +1078,7 @@ impl<'a, 'b: 'a> StatementParser<'a, 'b> {
};
if !ctx
.resolve_column(column)
- .table
+ .and_then(|column| column.table)
.is_some_and(|table| self.schema.tables.omnishards().contains_key(table))
{
continue;
@@ -1079,9 +1097,12 @@ impl<'a, 'b: 'a> StatementParser<'a, 'b> {
if *from != column {
continue;
}
+ let Some(to) = ctx.resolve_column(*to) else {
+ continue;
+ };
let shards = values
.iter()
- .map(|value| self.compute_shard_with_ctx(*to, value.clone(), ctx))
+ .map(|value| self.compute_shard_of_resolved_column(to, value.clone()))
.collect::<Result<Vec<_>, _>>()
.break_err()?;
// Resolve every value above so all pending lookups
@@ -1113,11 +1134,14 @@ impl<'a, 'b: 'a> StatementParser<'a, 'b> {
Node::ColumnRef(c) => {
let mut column = Column::try_from(c).break_err()?;
- // If column has no table, qualify with context table
+ // If the column doesn't have a table, we use the alias (or name) of `SearchContext::table`
+ // E.g., for id in FROM users u WHERE id = 1, we use u.id (not users.id), since `SearchContext::relations` stores the table under the alias u
+ // `resolve_column` then replaces u with users
if column.table().is_none()
&& let Some(ref table) = ctx.table
{
- column.qualify(*table);
+ column.table = Some(table.alias.unwrap_or(table.name));
+ column.schema = table.schema;
}
// In joins, a named sharding config can identify the owner
// of an otherwise unqualified key. Only consider tables in
@@ -1227,15 +1251,15 @@ impl<'a, 'b: 'a> StatementParser<'a, 'b> {
Some(array_elements_as_text)
}
- /// Compute shard with alias resolution from context.
- fn compute_shard_with_ctx(
+ /// Returns the shard that has the rows where `resolved_column` = `value`, e.g., the shard corresponding to 5 for users.id = 5, or None if `resolved_column` isn't a sharding key
+ /// `resolved_column` needs the table name (users.id), not the alias (u.id) (since the sharding config lists table names)
+ /// Callers resolve the column (`resolve_column`) once before the loop over the values (e.g., 1, 2 and 3 in users.id IN (1, 2, 3)).
+ /// Resolving looks up the alias of the column (u in u.id) in the hash map `SearchContext::relations`. Doing that per value (e.g., 1,000 times for a subquery that returns 1,000 rows) adds about 5% to the CPU instructions of those subqueries
+ fn compute_shard_of_resolved_column(
&mut self,
- column: Column<'a>,
+ resolved_column: Column<'a>,
value: Value<'_>,
- ctx: &SearchContext<'a>,
) -> Result<Option<Shard>, Error> {
- let resolved_column = ctx.resolve_column(column);
-
let shard = self.compute_shard(resolved_column, value.clone())?;
if let Some(ref shard) = shard {
self.record_sharding_key(shard, resolved_column, &value);
@@ -3243,4 +3267,36 @@ mod test {
"DELETE from omnisharded table should not be sharded"
);
}
+
+ // This tests that the router doesn't pick a shard by the columns belonging to a derived table that has the name of the sharded table `sharded`
+ // E.g., in SELECT * FROM (SELECT 1 AS id) sharded WHERE sharded.id = 1, sharded.id is a column belonging to the derived table (not the sharding key)
+ #[test]
+ fn test_derived_table_named_like_a_sharded_table() {
+ let table = "SELECT * FROM sharded WHERE sharded.id = 1";
+ let derived_table = "SELECT * FROM (SELECT 1 AS id) sharded WHERE sharded.id = 1";
+
+ assert!(run_test(table, None).unwrap().is_some());
+ assert_eq!(run_test(derived_table, None).unwrap(), None);
+ }
+
+ // Same test as `test_derived_table_named_like_a_sharded_table, but now for CTE that has name of the sharded table `sharded` since FROM sharded then refers to the CTE
+ #[test]
+ fn test_cte_named_like_a_sharded_table() {
+ let table = "SELECT * FROM sharded WHERE id = 1";
+ let cte = "WITH sharded AS (SELECT 1 AS id) SELECT * FROM sharded WHERE id = 1";
+
+ assert!(run_test(table, None).unwrap().is_some());
+ assert_eq!(run_test(cte, None).unwrap(), None);
+ }
+
+ // If we have a sharding config that sets a column without a table (column = "tenant_id" without name in [[sharded_tables]]), all tables with a tenant_id column are sharded by tenant_id
+ // This tests that the router doesn't pick a shard by tenant_id columns belonging to derived tables (e.g., s.tenant_id in FROM (SELECT 1 AS tenant_id) s)
+ #[test]
+ fn test_column_only_config_derived_table() {
+ let table = "SELECT * FROM t WHERE tenant_id = 1";
+ let derived_table = "SELECT * FROM (SELECT 1 AS tenant_id) s WHERE s.tenant_id = 1";
+
+ assert!(run_test_column_only(table, None).unwrap().is_some());
+ assert_eq!(run_test_column_only(derived_table, None).unwrap(), None);
+ }
}
^ permalink raw [nested|flat] 6+ messages in thread
* Re: [pgdogdev/pgdog] PR #2064: fix: don't route by cols belonging to CTEs / derived tables if same name as sharded table & key
@ 2026-11-18 20:10 ` "codecov[bot] (bot) (@codecov[bot])" <noreply+codecov[bot]@github.com>
4 siblings, 0 replies; 6+ messages in thread
From: codecov[bot] (bot) (@codecov[bot]) @ 2026-11-18 20:10 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
## [Codecov](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2064?dropdown=coverage&src=pr&el=h1&utm_me...) Report
:x: Patch coverage is `91.66667%` with `6 lines` in your changes missing coverage. Please review.
| [Files with missing lines](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2064?dropdown=coverage&src=pr&el=tree&utm_...) | Patch % | Lines |
|---|---|---|
| [pgdog/src/frontend/router/parser/statement.rs](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2064?src=pr&el=tree&utm_medium=referral&ut...) | 91.66% | [6 Missing :warning: ](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2064?src=pr&el=tree&utm_medium=referral&ut...) |
:loudspeaker: Thoughts on this report? [Let us know!](https://github.com/codecov/feedback/issues/255)
^ permalink raw [nested|flat] 6+ messages in thread
* Re: [pgdogdev/pgdog] PR #2064: fix: don't route by cols belonging to CTEs / derived tables if same name as sharded table & key
@ 2026-11-18 20:49 ` "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
4 siblings, 0 replies; 6+ messages in thread
From: github-actions[bot] (bot) (@github-actions[bot]) @ 2026-11-18 20:49 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
<!-- metal-bench -->
**Metal bench (arm64) failed.** 17 of 520 measures outside bounds.
| suite | throughput | cpu/txn | user cycles/txn | user instr/txn | kernel cycles/txn | kernel instr/txn | ctx/txn |
|---|---|---|---|---|---|---|---|
| pooler | | 18.88 (-0.3% ± 0.9%) | 17,735 (+0.2% ± 0.5%) | 31,381 (-0.0% ± 0.04%) | 30,528 (-0.7% ± 1.5%) | 51,149 (+0.1% ± 0.6%) | |
| lb | | 19.36 (+0.2% ± 0.5%) | 18,265 (-0.1% ± 0.8%) | 32,294 (-0.0% ± 0.07%) | 31,030 (+0.4% ± 1.2%) | 52,240 (-0.2% ± 0.3%) | |
| sharding-w2 | | 32.41 (+0.9% ± 1.8%) | 42,669 (-0.1% ± 1.3%) | 94,934 (+0.0% ± 0.02%) | 41,592 (+2.0% ± 2.8%) | 53,706 (-0.0% ± 0.04%) | 1.549 (-0.1% ± 0.2%) |
| sharding-w8 | 244,950 (+0.3% ± 0.2%) | 29.21 (-0.1% ± 0.7%) | 44,122 (+0.2% ± 2.4%) | 50,494 (+0.0% ± 0.03%) | 36,554 (-0.3% ± 1.2%) | 42,360 (-0.0% ± 0.09%) | 0.1672 (-1.1% ± 3.6%) |
<details><summary>Counters per txn</summary>
| suite | frontend stall | l1i refill | backend stall |
|---|---|---|---|
| pooler | 7,358 (+0.4% ± 1.7%) | 2,640 (+0.2% ± 0.7%) | |
| lb | 7,657 (-0.4% ± 2.1%) | 2,678 (-0.1% ± 0.8%) | |
| sharding-w2 | 17,293 (-0.6% ± 3.4%) | 4,109 (-0.5% ± 1.6%) | |
| sharding-w8 | 20,428 (+1.2% ± 8.6%) | 4,203 (+1.0% ± 4.6%) | |
</details>
| benchmark | measure | result | baseline | change | bound |
|---|---|---|---|---|---|
| extended_cached::cte | alloc_bytes | 1,264 | 1,156 | **+9.34% ± 0.00%** | +1% |
| extended_cached::cte | instructions | 23,932 | 23,657 | **+1.16% ± 0.00%** | +1% |
| extended_cached::join | alloc_bytes | 1,572 | 1,540 | **+2.08% ± 0.00%** | +1% |
| extended_cold::cte | alloc_bytes | 1,815 | 1,707 | **+6.33% ± 0.00%** | +1% |
| extended_cold::join | alloc_bytes | 2,150 | 2,118 | **+1.51% ± 0.00%** | +1% |
| extended_evicting::cte | alloc_bytes | 1,775 | 1,667 | **+6.48% ± 0.00%** | +1% |
| extended_evicting::cte | instructions | 48,753 | 48,187 | **+1.17% ± 0.00%** | +1% |
| extended_evicting::join | alloc_bytes | 2,110 | 2,078 | **+1.54% ± 0.00%** | +1% |
| full_extended_cached::cte | alloc_bytes | 7,459 | 7,351 | **+1.47% ± 0.00%** | +1% |
| full_named_cached::aggregate | instructions | 136,186 | 134,377 | **+1.35% ± 0.00%** | +1% |
| full_named_cached::cte | alloc_bytes | 7,336 | 7,228 | **+1.49% ± 0.00%** | +1% |
| full_named_cached::update_by_key | instructions | 61,805 | 60,503 | **+2.15% ± 0.00%** | +1% |
| full_simple::cte | alloc_bytes | 7,498 | 7,390 | **+1.46% ± 0.00%** | +1% |
| full_simple::order_limit_offset | instructions | 195,469 | 192,751 | **+1.41% ± 0.00%** | +1% |
| simple_cold::cte | alloc_bytes | 1,775 | 1,667 | **+6.48% ± 0.00%** | +1% |
| simple_cold::cte | instructions | 47,013 | 46,526 | **+1.05% ± 0.00%** | +1% |
| simple_cold::join | alloc_bytes | 2,110 | 2,078 | **+1.54% ± 0.00%** | +1% |
Microbench: 490 of 507 measures within bounds.
<details><summary>Hot code per txn</summary>
| path | base | head | change |
|---|---|---|---|
| pooler | 113.9 KB | 112.3 KB | **-1.6 KB** |
| sharding-w2 | 168.4 KB | 168.9 KB | **+0.6 KB** |
pooler:
| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::client::query_engine | 20.1 KB | 20.0 KB | -0.1 KB |
| tokio | 18.6 KB | 18.2 KB | -0.4 KB |
| pgdog::backend::pool | 17.5 KB | 17.4 KB | -0.1 KB |
| aws-lc (TLS crypto) | 10.4 KB | 10.2 KB | -0.2 KB |
| rustls (TLS) | 8.5 KB | 8.3 KB | -0.2 KB |
| pgdog::frontend (other) | 7.9 KB | 7.8 KB | -0.1 KB |
| pgdog::backend::server | 7.6 KB | 7.6 KB | -0.1 KB |
| other | 6.1 KB | 6.1 KB | +0.0 KB |
| everything else | 17.1 KB | 16.6 KB | -0.4 KB |
| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 5.5 KB | -0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 5.2 KB | +0.0 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 4.0 KB | -0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 3.4 KB | -0.1 KB |
| `tokio::runtime::task::raw::poll` | 3.3 KB | +0.1 KB |
sharding-w2:
| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::router | 29.2 KB | 29.3 KB | +0.1 KB |
| pgdog::frontend::client::query_engine | 21.1 KB | 21.3 KB | +0.2 KB |
| tokio | 18.7 KB | 18.6 KB | -0.1 KB |
| pgdog::backend::pool | 18.2 KB | 18.3 KB | +0.1 KB |
| aws-lc (TLS crypto) | 13.3 KB | 13.2 KB | -0.1 KB |
| other | 9.9 KB | 9.9 KB | +0.0 KB |
| pgdog::frontend (other) | 9.1 KB | 8.9 KB | -0.1 KB |
| pgdog::backend::server | 8.0 KB | 8.1 KB | +0.1 KB |
| everything else | 40.9 KB | 41.1 KB | +0.2 KB |
| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 6.4 KB | -0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 5.2 KB | +0.1 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 4.0 KB | +0.1 KB |
| `pgdog::frontend::router::parser::query::select::<impl pgdog::frontend::router::parser::que` | 3.3 KB | +0.0 KB |
| `tokio::runtime::task::raw::poll` | 3.2 KB | +0.1 KB |
</details>
<details><summary>lb flamegraph vs 8cfcf31f</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-router-derived-table-aliases/39d917121f306cddf...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-router-derived-table-aliases/39d917121f306cddf...).
</details>
<details><summary>microbench flamegraph vs 8cfcf31f</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-router-derived-table-aliases/39d917121f306cddf...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-router-derived-table-aliases/39d917121f306cddf...).
</details>
<details><summary>pooler flamegraph vs 8cfcf31f</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-router-derived-table-aliases/39d917121f306cddf...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-router-derived-table-aliases/39d917121f306cddf...).
</details>
<details><summary>sharding-w2 flamegraph vs 8cfcf31f</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-router-derived-table-aliases/39d917121f306cddf...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-router-derived-table-aliases/39d917121f306cddf...).
</details>
<details><summary>sharding-w8 flamegraph vs 8cfcf31f</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-router-derived-table-aliases/39d917121f306cddf...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-router-derived-table-aliases/39d917121f306cddf...).
</details>
<details><summary>Logs</summary>
[lb](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/jk-router-derived-table...), [microbench](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/jk-router-derived-table...), [pooler](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/jk-router-derived-table...), [sharding-w2](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/jk-router-derived-table...), [sharding-w8](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/jk-router-derived-table...), [verdict.json](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/jk-router-derived-table...)
</details>
Compared 39d91712 with 8cfcf31f on m8g.metal-24xl (spot).
^ permalink raw [nested|flat] 6+ messages in thread
* Re: [pgdogdev/pgdog] PR #2064: fix: don't route by cols belonging to CTEs / derived tables if same name as sharded table & key
@ 2026-11-18 23:05 ` "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
4 siblings, 0 replies; 6+ messages in thread
From: github-actions[bot] (bot) (@github-actions[bot]) @ 2026-11-18 23:05 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
<!-- metal-bench-x86 -->
**Metal bench (x86) failed.** 16 of 520 measures outside bounds.
| suite | throughput | cpu/txn | user cycles/txn | user instr/txn | kernel cycles/txn | kernel instr/txn | ctx/txn |
|---|---|---|---|---|---|---|---|
| pooler | | 34.92 (+0.2% ± 0.2%) | 22,865 (+0.3% ± 0.9%) | 32,734 (-0.0% ± 0.02%) | 58,400 (+0.2% ± 0.4%) | 38,531 (-0.1% ± 0.3%) | |
| lb | | 35.89 (-0.2% ± 0.2%) | 24,000 (-0.3% ± 0.7%) | 33,700 (+0.0% ± 0.02%) | 59,225 (-0.2% ± 0.1%) | 39,229 (+0.0% ± 0.3%) | |
| sharding-w2 | | 56.61 (-0.1% ± 1.6%) | 52,785 (+0.1% ± 1.9%) | 98,604 (+0.0% ± 0.01%) | 73,290 (-0.2% ± 2.7%) | 43,065 (-0.0% ± 0.1%) | 1.391 (+0.0% ± 0.5%) |
| sharding-w8 | 146,408 (-0.8% ± 2.1%) | 48.56 (+0.4% ± 1.4%) | 57,965 (+0.1% ± 1.9%) | 54,108 (-0.0% ± 0.2%) | 65,274 (+0.6% ± 1.2%) | 34,220 (+0.0% ± 0.2%) | 0.1723 (-0.1% ± 1.9%) |
<details><summary>Counters per txn</summary>
| suite | frontend stall | l1i refill | backend stall |
|---|---|---|---|
| pooler | 74,477 (+0.6% ± 1.0%) | 3,629 (+0.1% ± 0.4%) | |
| lb | 77,395 (-0.5% ± 0.4%) | 3,748 (-0.0% ± 0.3%) | |
| sharding-w2 | 159,547 (+0.0% ± 4.8%) | 6,062 (+0.2% ± 0.8%) | |
| sharding-w8 | 120,601 (-0.3% ± 1.7%) | 4,892 (+0.1% ± 0.5%) | |
</details>
| benchmark | measure | result | baseline | change | bound |
|---|---|---|---|---|---|
| extended_cached::cte | alloc_bytes | 1,296 | 1,180 | **+9.83% ± 0.00%** | +1% |
| extended_cached::cte | instructions | 24,219 | 23,906 | **+1.31% ± 0.00%** | +1% |
| extended_cached::join | alloc_bytes | 1,596 | 1,564 | **+2.05% ± 0.00%** | +1% |
| extended_cold::cte | alloc_bytes | 1,847 | 1,731 | **+6.70% ± 0.00%** | +1% |
| extended_cold::cte | instructions | 48,591 | 47,994 | **+1.24% ± 0.00%** | +1% |
| extended_cold::join | alloc_bytes | 2,174 | 2,142 | **+1.49% ± 0.00%** | +1% |
| extended_evicting::cte | alloc_bytes | 1,807 | 1,691 | **+6.86% ± 0.00%** | +1% |
| extended_evicting::join | alloc_bytes | 2,134 | 2,102 | **+1.52% ± 0.00%** | +1% |
| full_extended_cached::cte | alloc_bytes | 7,491 | 7,375 | **+1.57% ± 0.00%** | +1% |
| full_named_cached::aggregate | instructions | 142,462 | 140,677 | **+1.27% ± 0.00%** | +1% |
| full_named_cached::cte | alloc_bytes | 7,368 | 7,252 | **+1.60% ± 0.00%** | +1% |
| full_simple::cte | alloc_bytes | 7,530 | 7,414 | **+1.56% ± 0.00%** | +1% |
| full_simple::cte | instructions | 106,116 | 104,759 | **+1.29% ± 0.00%** | +1% |
| simple_cold::cte | alloc_bytes | 1,807 | 1,691 | **+6.86% ± 0.00%** | +1% |
| simple_cold::cte | instructions | 47,847 | 47,368 | **+1.01% ± 0.00%** | +1% |
| simple_cold::join | alloc_bytes | 2,134 | 2,102 | **+1.52% ± 0.00%** | +1% |
Microbench: 491 of 507 measures within bounds.
<details><summary>Microbench latency: 4 benches changed more than 3%</summary>
| benchmark | latency | baseline | change | instructions change |
|---|---|---|---|---|
| extended_cached::group_by | 2,888 ns | 2,742 ns | +5.33% ± 0.77% | +0.03% ± 0.00% |
| extended_cached::subquery | 3,164 ns | 3,020 ns | +4.78% ± 4.69% | +0.05% ± 0.00% |
| full_simple::aggregate | 26,788 ns | 27,974 ns | -4.23% ± 1.75% | +0.09% ± 0.00% |
| extended_cold::subquery | 6,148 ns | 5,914 ns | +3.97% ± 2.33% | +0.66% ± 0.00% |
</details>
<details><summary>Hot code per txn</summary>
| path | base | head | change |
|---|---|---|---|
| pooler | 106.4 KB | 108.3 KB | **+1.9 KB** |
| sharding-w2 | 166.2 KB | 165.8 KB | **-0.4 KB** |
pooler:
| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::client::query_engine | 20.8 KB | 21.2 KB | +0.4 KB |
| tokio | 17.4 KB | 17.4 KB | +0.1 KB |
| pgdog::backend::pool | 16.2 KB | 17.0 KB | +0.8 KB |
| pgdog::frontend (other) | 8.6 KB | 8.7 KB | +0.1 KB |
| pgdog::backend::server | 7.2 KB | 7.5 KB | +0.3 KB |
| rustls (TLS) | 6.9 KB | 7.2 KB | +0.3 KB |
| aws-lc (TLS crypto) | 7.6 KB | 7.2 KB | -0.4 KB |
| other | 5.8 KB | 5.9 KB | +0.1 KB |
| everything else | 15.9 KB | 16.1 KB | +0.2 KB |
| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 6.6 KB | +0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 6.6 KB | +0.2 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 4.2 KB | +0.0 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 3.8 KB | +0.0 KB |
| `tokio::runtime::task::raw::poll` | 3.6 KB | +0.0 KB |
sharding-w2:
| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::router | 27.4 KB | 27.2 KB | -0.2 KB |
| pgdog::frontend::client::query_engine | 22.9 KB | 23.1 KB | +0.2 KB |
| tokio | 18.1 KB | 17.9 KB | -0.1 KB |
| pgdog::backend::pool | 18.1 KB | 17.8 KB | -0.3 KB |
| other | 11.0 KB | 10.4 KB | -0.6 KB |
| aws-lc (TLS crypto) | 10.3 KB | 10.3 KB | +0.0 KB |
| pgdog::frontend (other) | 10.1 KB | 10.1 KB | +0.1 KB |
| jemalloc | 9.2 KB | 9.5 KB | +0.2 KB |
| everything else | 39.1 KB | 39.4 KB | +0.3 KB |
| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 7.6 KB | +0.0 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 6.2 KB | +0.2 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 4.5 KB | +0.0 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 3.9 KB | +0.1 KB |
| `pgdog::frontend::router::parser::query::select::<impl pgdog::frontend::router::parser::que` | 3.9 KB | +0.0 KB |
</details>
<details><summary>lb flamegraph vs 8cfcf31f</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-router-derived-table-aliases/39d917121f306...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-router-derived-table-aliases/39d917121f306...).
</details>
<details><summary>microbench flamegraph vs 8cfcf31f</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-router-derived-table-aliases/39d917121f306...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-router-derived-table-aliases/39d917121f306...).
</details>
<details><summary>pooler flamegraph vs 8cfcf31f</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-router-derived-table-aliases/39d917121f306...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-router-derived-table-aliases/39d917121f306...).
</details>
<details><summary>sharding-w2 flamegraph vs 8cfcf31f</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-router-derived-table-aliases/39d917121f306...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-router-derived-table-aliases/39d917121f306...).
</details>
<details><summary>sharding-w8 flamegraph vs 8cfcf31f</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-router-derived-table-aliases/39d917121f306...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-router-derived-table-aliases/39d917121f306...).
</details>
<details><summary>Logs</summary>
[lb](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/jk-router-derived-t...), [microbench](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/jk-router-derived-t...), [pooler](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/jk-router-derived-t...), [sharding-w2](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/jk-router-derived-t...), [sharding-w8](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/jk-router-derived-t...), [verdict.json](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/jk-router-derived-t...)
</details>
Compared 39d91712 with 8cfcf31f on c7a.metal-48xl (spot).
^ permalink raw [nested|flat] 6+ messages in thread
* Re: [pgdogdev/pgdog] PR #2064: fix: don't route by cols belonging to CTEs / derived tables if same name as sharded table & key
@ 2026-11-19 13:56 ` "jkaczman (@jkaczman)" <noreply+jkaczman@github.com>
4 siblings, 0 replies; 6+ messages in thread
From: jkaczman (@jkaczman) @ 2026-11-19 13:56 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
@jkaczman merged this (commit 6b0e24d7f6b8).
^ permalink raw [nested|flat] 6+ messages in thread
* Re: [pgdogdev/pgdog] PR #2064: fix: don't route by cols belonging to CTEs / derived tables if same name as sharded table & key
@ 2026-11-19 13:56 ` "jkaczman (@jkaczman)" <noreply+jkaczman@github.com>
4 siblings, 0 replies; 6+ messages in thread
From: jkaczman (@jkaczman) @ 2026-11-19 13:56 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
@jkaczman closed this.
^ permalink raw [nested|flat] 6+ messages in thread
end of thread, other threads:[~2026-11-19 13:56 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-11-18 18:24 [pgdogdev/pgdog] PR #2064: fix: don't route by cols belonging to CTEs / derived tables if same name as sharded table & key "jkaczman (@jkaczman)" <noreply+jkaczman@github.com>
2026-11-18 20:10 ` "codecov[bot] (bot) (@codecov[bot])" <noreply+codecov[bot]@github.com>
2026-11-18 20:49 ` "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
2026-11-18 23:05 ` "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
2026-11-19 13:56 ` "jkaczman (@jkaczman)" <noreply+jkaczman@github.com>
2026-11-19 13:56 ` "jkaczman (@jkaczman)" <noreply+jkaczman@github.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox