pg.ddx.io pgdogdev/pgdog GitHub issues and pull requests
help / color / mirror / Atom feed[pgdogdev/pgdog] PR #2061: feat: support array columns encountered in subqueries
6+ messages / 3 participants
[nested] [flat]
* [pgdogdev/pgdog] PR #2061: feat: support array columns encountered in subqueries
@ 2026-11-18 17:16 "jkaczman (@jkaczman)" <noreply+jkaczman@github.com>
0 siblings, 0 replies; 6+ messages in thread
From: jkaczman (@jkaczman) @ 2026-11-18 17:16 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
When we rewrite a cross-shard subquery, we send the rows of the subquery back to the shards as an `unnest(..)` array per column. Building off of #2060, this now introduces support to handle array columns (e.g., `text[]`). Previously, we returned an unsupported error when encountering these types in subqueries.
We can't send an array column via an `unnest(..)` array as that would resolve to be a 2D array like `{{a,b},NULL,{c}}`. Postgres doesn't allow that **specific** 2D array because the inner arrays of 2D arrays are required to all have the same length.
Instead, we send the elements of all rows as one flat array. How is this possible? We basically give the starting positions and lengths of each inner array within the outer array as context within the statement. E.g., for the rows `{a,b}`, NULL and `{c}`, we send:
- the flat array `{a,b,c}` as parameter `$2`
- the positions `{0,2,2}` and lengths `{2,NULL,1}` as `unnest(..)` arguments, which become the columns `c2` and `c3` of `unnest(..) u`
The rewritten statement then uses the lengths and positions of each inner array to take the inner array for each row back out of the flat array via slice, `($2::pg_catalog._text)[u.c2 + 1:u.c2 + u.c3]`.
Okay, what about 2D+ arrays, or ones that start at index 1 (e.g., `'[0:1]={1,2}'`)? These still emit an error, for now. It seems like these are rare in application schemas. A problem for later!
This is partially motivated by Mastodon: the accounts table in Mastodon (which is joined to most other tables) has a varchar[] column (attribution_domains)
inline patch 1: integration/rust/tests/integration/subqueries/mod.rs, integration/rust/tests/integration/subqueries/tests/select.sql, pgdog-postgres-types/src/array.rs and 3 more (+271/-46 lines) download inline-1.patch
diff --git a/integration/rust/tests/integration/subqueries/mod.rs b/integration/rust/tests/integration/subqueries/mod.rs
index b20969b98..bd7983f32 100644
--- a/integration/rust/tests/integration/subqueries/mod.rs
+++ b/integration/rust/tests/integration/subqueries/mod.rs
@@ -61,7 +61,7 @@ async fn setup_subqueries_suite(suffix: &str, first_time_running: bool) {
"DROP TABLE IF EXISTS {subpackages_table}, {suborders_table}, {subusers_table}, {substates_table}"
),
format!(
- "CREATE TABLE {subusers_table} (id BIGINT PRIMARY KEY, username text, status text, profile jsonb, signed_up date, last_ip inet)"
+ "CREATE TABLE {subusers_table} (id BIGINT PRIMARY KEY, username text, status text, profile jsonb, signed_up date, last_ip inet, tags varchar[], media_ids bigint[])"
),
format!("CREATE TABLE {substates_table} (state text PRIMARY KEY, shipping text)"),
format!(
@@ -81,13 +81,13 @@ async fn setup_subqueries_suite(suffix: &str, first_time_running: bool) {
format!("INSERT INTO {substates_table}(state, shipping) VALUES ('CA', 'quick')"),
format!("INSERT INTO {substates_table}(state, shipping) VALUES ('NC', 'not quick')"),
format!(
- "INSERT INTO {subusers_table}(id, username, status, profile, signed_up, last_ip) VALUES (1, 'test', 'VIP', '{{\"plan\": \"pro\"}}', '2026-01-01', '10.0.0.1')"
+ "INSERT INTO {subusers_table}(id, username, status, profile, signed_up, last_ip, tags, media_ids) VALUES (1, 'test', 'VIP', '{{\"plan\": \"pro\"}}', '2026-01-01', '10.0.0.1', '{{a,b}}', '{{1,2}}')"
),
format!(
- "INSERT INTO {subusers_table}(id, username, status, profile, signed_up, last_ip) VALUES (2, 'test2', 'regular', '{{\"plan\": \"free\"}}', '2026-02-01', '192.168.0.2')"
+ "INSERT INTO {subusers_table}(id, username, status, profile, signed_up, last_ip, tags, media_ids) VALUES (2, 'test2', 'regular', '{{\"plan\": \"free\"}}', '2026-02-01', '192.168.0.2', '{{}}', '{{}}')"
),
format!(
- "INSERT INTO {subusers_table}(id, username, status, profile, signed_up, last_ip) VALUES (3, 'test3', 'VIP', '{{\"plan\": \"pro\"}}', '2026-03-01', '10.0.0.3')"
+ "INSERT INTO {subusers_table}(id, username, status, profile, signed_up, last_ip, tags, media_ids) VALUES (3, 'test3', 'VIP', '{{\"plan\": \"pro\"}}', '2026-03-01', '10.0.0.3', '{{c,NULL}}', '{{3}}')"
),
format!("INSERT INTO {subusers_table}(id, username, status) VALUES (4, 'test4', NULL)"),
format!(
diff --git a/integration/rust/tests/integration/subqueries/tests/select.sql b/integration/rust/tests/integration/subqueries/tests/select.sql
index 78c0dd61d..084658f80 100644
--- a/integration/rust/tests/integration/subqueries/tests/select.sql
+++ b/integration/rust/tests/integration/subqueries/tests/select.sql
@@ -208,6 +208,13 @@ SELECT * FROM (
SELECT * FROM subusers WHERE status IS NOT NULL
) s ORDER BY s.id DESC LIMIT 2;
+-- case
+-- what: SELECT * FROM (<sq>) s, <sq> returns an array column
+-- route select: 0,1
+-- route select: any
+-- sql: SELECT * FROM (SELECT u.c1, ($2::pg_catalog._varchar)[u.c2 + 1:u.c2 + u.c3] FROM unnest($1::pg_catalog._int8, $3::pg_catalog._int4, $4::pg_catalog._int4) u(c1, c2, c3)) s(id, tags) ORDER BY id
+SELECT * FROM (SELECT id, tags FROM subusers) s ORDER BY id;
+
-- case
-- what: Derived table with an inner-table that returns an empty response.
-- Want to make sure that the OIDs returned are correct (if present)
diff --git a/pgdog-postgres-types/src/array.rs b/pgdog-postgres-types/src/array.rs
index 1173dd4d7..79ac3536c 100644
--- a/pgdog-postgres-types/src/array.rs
+++ b/pgdog-postgres-types/src/array.rs
@@ -192,6 +192,43 @@ impl Array {
buf.freeze()
}
+ /// Splits `array` (array in binary format) into the values of the elements of `array`.
+ /// E.g., for bigint[] {1,NULL,3}, we return [8 bytes of 1], [None], and [8 bytes of 3]
+ /// Returns None for arrays with more than one dimension (e.g., {{1,2},{3,4}}) or that don't start at index 1 (e.g., '[0:1]={1,2}')
+ pub fn binary_elements(array: &[u8]) -> Option<Vec<Option<&[u8]>>> {
+ let read_i32 = |pos: usize| -> Option<i32> {
+ Some(i32::from_be_bytes(
+ array.get(pos..pos + 4)?.try_into().ok()?,
+ ))
+ };
+
+ let dimensions = read_i32(0)?;
+ let mut pos = 12;
+ let element_count = match dimensions {
+ 0 => 0,
+ 1 if read_i32(16)? == 1 => {
+ pos += 8;
+ usize::try_from(read_i32(12)?).ok()?
+ }
+ _ => return None,
+ };
+
+ let mut elements = Vec::with_capacity(element_count);
+ for _ in 0..element_count {
+ let len = read_i32(pos)?;
+ pos += 4;
+ if len < 0 {
+ elements.push(None);
+ } else {
+ let end = pos + len as usize;
+ elements.push(Some(array.get(pos..end)?));
+ pos = end;
+ }
+ }
+
+ Some(elements)
+ }
+
fn encode_binary(&self) -> Result<Bytes, Error> {
let mut buf = BytesMut::with_capacity(12 + 8 + self.elements.len() * 8);
// Use ndims=0 only for truly empty arrays with default bounds.
@@ -485,6 +522,42 @@ fn decode_binary(bytes: &[u8], expected_element_oid: i32) -> Result<Array, Error
#[cfg(test)]
mod tests {
+ #[test]
+ fn test_binary_elements() {
+ let one = 1i64.to_be_bytes();
+ let three = 3i64.to_be_bytes();
+ let elements = [Some(one.as_slice()), None, Some(three.as_slice())];
+
+ // Splitting {1,NULL,3} returns the elements
+ let array = Array::binary_from_encoded_elements(20, elements.into_iter());
+ assert_eq!(Array::binary_elements(&array), Some(elements.to_vec()));
+
+ // {}, no elements
+ let empty = Array::binary_from_encoded_elements(20, std::iter::empty());
+ assert_eq!(Array::binary_elements(&empty), Some(vec![]));
+
+ // Arrays rejected; can't rebuild from just the elements
+ assert_eq!(
+ Array::binary_elements(&binary_header(&[(2, 1), (1, 1)])),
+ None
+ );
+
+ assert_eq!(Array::binary_elements(&binary_header(&[(2, 0)])), None); // '[0:1]={1,3}'
+ }
+
+ /// Returns the header of a bigint[] in binary format (without the elements) for `dimensions`, which is given to us as (length, first index)
+ /// E.g., for [(2, 1), (1, 1)], we return the header of a 2 by 1 array such as {{1},{3}}.
+ fn binary_header(dimensions: &[(i32, i32)]) -> Vec<u8> {
+ let mut header = [dimensions.len() as i32, 0, 20].to_vec();
+ for (length, first_index) in dimensions {
+ header.extend([*length, *first_index]);
+ }
+ header
+ .iter()
+ .flat_map(|value| value.to_be_bytes())
+ .collect()
+ }
+
use super::*;
use crate::ToDataRowColumn;
diff --git a/pgdog-postgres-types/src/datum.rs b/pgdog-postgres-types/src/datum.rs
index 15fcf29bd..6ef236b55 100644
--- a/pgdog-postgres-types/src/datum.rs
+++ b/pgdog-postgres-types/src/datum.rs
@@ -402,6 +402,18 @@ impl DataType {
None
}
+ /// Returns the OID of the elements of the built-in array type `array_oid`, e.g., 20 (bigint) for bigint[] (1016).
+ /// Returns None for types that aren't arrays of built-in types (e.g., bigint or arrays of enums).
+ pub fn array_element_oid(array_oid: i32) -> Option<i32> {
+ for (element_oid, oid, _) in BUILTIN_ARRAY_TYPES {
+ if oid == array_oid {
+ return Some(element_oid);
+ }
+ }
+
+ None
+ }
+
pub fn pg_catalog_type_to_datatype(pg_catalog_type: &str) -> Option<Self> {
Some(match pg_catalog_type {
"int8" => DataType::Bigint,
diff --git a/pgdog/src/frontend/client/query_engine/multi_step/subquery/clause/from_clause.rs b/pgdog/src/frontend/client/query_engine/multi_step/subquery/clause/from_clause.rs
index b4603bc52..7279ab6f9 100644
--- a/pgdog/src/frontend/client/query_engine/multi_step/subquery/clause/from_clause.rs
+++ b/pgdog/src/frontend/client/query_engine/multi_step/subquery/clause/from_clause.rs
@@ -110,8 +110,6 @@ impl<'a> SubqueryMulti<'a> {
data_rows: &[DataRow],
row_description: &RowDescription,
) -> Unique<'mem, &'mem SelectStmt> {
- let base_param_len = old_bind.params_raw().len() as i32;
-
let original_targets = match original_ast {
Node::SelectStmt(original_select)
if original_select.target_list().len() == row_description.fields.len() =>
@@ -139,6 +137,100 @@ impl<'a> SubqueryMulti<'a> {
continue;
}
+ // Why can't we send an array column (e.g., tags text[]) as an unnest(..) argument like the other columns?
+ // Because that argument would be a 2D array (e.g., {{a,b},NULL,{c}}), but the inner arrays of 2D arrays are **required to be the same length**
+ // Also, unnest(..) on 2D array returns the elements of the array flattened, not one row per inner array
+ //
+ // So, instead,
+ // - we send the elements of all rows in a singular flat array
+ // - for each row, we also send where the elements of the row start in the flat array (position), and how many elements the row has (length)
+ // E.g., for rows (1, {a,b}), (2, NULL) and (3, {c}), we send the flat array {a,b,c} as a parameter, the positions {0,2,2} and the lengths {2,NULL,1} as unnest(..) arguments.
+ //
+ // The SELECT that we're building here then reads the rows of unnest(..) u.
+ // For each row, the SELECT slices the flat array from position + 1 to position + length (we add 1, since Postgres arrays start at index 1):
+ // {a,b,c}[0 + 1:0 + 2] = {a,b} for row 1, NULL for row 2 (slice with a NULL bound is NULL), and {a,b,c}[2 + 1:2 + 1] = {c} for row 3
+ if let Some(element_oid) = DataType::array_element_oid(field.type_oid) {
+ let array_type_name = DataType::array_type_name(field.type_oid)
+ .expect("array_element_oid only returns elements of built-in array types");
+
+ let (flat, positions, lengths) = {
+ let mut flat = Vec::new();
+ let mut positions = Vec::with_capacity(data_rows.len());
+ let mut lengths = Vec::with_capacity(data_rows.len());
+
+ for row in data_rows {
+ positions.push(Some((flat.len() as i32).to_be_bytes()));
+
+ match row.get_raw(idx).filter(|value| !value.is_null) {
+ Some(value) => {
+ let elements = Array::binary_elements(value.data.as_ref()).expect(
+ "evaluate_and_rewrite_outer_statement checked the arrays",
+ );
+ lengths.push(Some((elements.len() as i32).to_be_bytes()));
+ flat.extend(elements);
+ }
+ None => lengths.push(None),
+ }
+ }
+
+ (flat, positions, lengths)
+ };
+
+ let flat_param = push_array_param(old_bind, element_oid, flat.into_iter());
+
+ let position_column = {
+ let position_column = format!("c{}", param_refs.len() + 1);
+ param_refs.push(push_unnest_argument(
+ mem,
+ old_bind,
+ INT4_OID,
+ "_int4",
+ positions
+ .iter()
+ .map(|position| position.as_ref().map(|bytes| bytes.as_slice())),
+ ));
+ unnest_column_names
+ .push(mem.make_string(Some(position_column.as_str())).uncast());
+ position_column
+ };
+
+ let length_column = {
+ let length_column = format!("c{}", param_refs.len() + 1);
+ param_refs.push(push_unnest_argument(
+ mem,
+ old_bind,
+ INT4_OID,
+ "_int4",
+ lengths
+ .iter()
+ .map(|length| length.as_ref().map(|bytes| bytes.as_slice())),
+ ));
+ unnest_column_names
+ .push(mem.make_string(Some(length_column.as_str())).uncast());
+ length_column
+ };
+
+ let slice_target = {
+ let sql = format!(
+ "SELECT (${flat_param}::pg_catalog.{array_type_name})[u.{position_column} + 1 : u.{position_column} + u.{length_column}]"
+ );
+ let parsed = pg_raw_parse::parse(&sql).expect("we build valid SQL");
+ let Some(Node::SelectStmt(select)) = parsed.stmts().next() else {
+ unreachable!("we parse a SELECT");
+ };
+ let slice = select
+ .target_list()
+ .into_iter()
+ .next()
+ .expect("the SELECT has one target")
+ .val();
+ mem.make_res_target(None, mem.empty(), mem.make_unique(slice))
+ };
+
+ select_targets.push(slice_target);
+ continue;
+ }
+
let unnest_column_name = format!("c{}", param_refs.len() + 1);
select_targets.push(
mem.make_res_target(
@@ -158,15 +250,6 @@ impl<'a> SubqueryMulti<'a> {
.and_then(DataType::array_type_name)
.expect("evaluate_and_rewrite_outer_statement checked the column types");
- let node = {
- let mut node = mem.make_node::<nodes::ParamRef>();
-
- node.as_mut()
- .set_number(base_param_len + param_refs.len() as i32 + 1);
-
- node
- };
-
// We build one array for each column that the subquery returned.
// E.g., if SELECT id, status FROM users returns the rows (1, 'VIP'), (2, NULL) and (3, 'VIP'), we build
// - {1,2,3} for id
@@ -177,21 +260,13 @@ impl<'a> SubqueryMulti<'a> {
.map(|value| value.data.as_ref())
});
- let array = Array::binary_from_encoded_elements(field.type_oid, elements);
-
- old_bind.push_param(Parameter::from_bytes(array), Format::Binary);
-
- let node = mem
- .make_type_cast(
- node.uncast(),
- mem.make_list(&[
- mem.make_string(Some("pg_catalog")),
- mem.make_string(Some(array_type_name)),
- ]),
- )
- .uncast();
-
- param_refs.push(node);
+ param_refs.push(push_unnest_argument(
+ mem,
+ old_bind,
+ field.type_oid,
+ array_type_name,
+ elements,
+ ));
}
let list_of_param_refs = mem.make_list(param_refs.as_slice());
@@ -269,3 +344,46 @@ impl<'a> SubqueryMulti<'a> {
new_alias
}
}
+
+const INT4_OID: i32 = 23;
+
+/// Adds the values `elements` to `bind` as an array parameter, and returns the number of that parameter
+/// E.g., for bigint values 1, 2 and 3 (`element_oid` 20), we add the array {1,2,3} as $2, and return 2
+fn push_array_param<'e>(
+ bind: &mut Bind,
+ element_oid: i32,
+ elements: impl ExactSizeIterator<Item = Option<&'e [u8]>> + Clone,
+) -> i32 {
+ bind.push_param(
+ Parameter::from_bytes(Array::binary_from_encoded_elements(element_oid, elements)),
+ Format::Binary,
+ );
+ bind.params_raw().len() as i32
+}
+
+/// Adds the values `elements` to `bind` as an array parameter (with `push_array_param`), and returns the argument of unnest(..) that reads that parameter
+/// E.g., for bigint values 1, 2 and 3, we add the array {1,2,3} as $2, and return $2::pg_catalog._int8
+fn push_unnest_argument<'mem, 'e>(
+ mem: MemoryToken<'mem>,
+ bind: &mut Bind,
+ element_oid: i32,
+ array_type_name: &str,
+ elements: impl ExactSizeIterator<Item = Option<&'e [u8]>> + Clone,
+) -> Unique<'mem, Node<'mem>> {
+ let param_ref = {
+ let mut param_ref = mem.make_node::<nodes::ParamRef>();
+ param_ref
+ .as_mut()
+ .set_number(push_array_param(bind, element_oid, elements));
+ param_ref
+ };
+
+ mem.make_type_cast(
+ param_ref.uncast(),
+ mem.make_list(&[
+ mem.make_string(Some("pg_catalog")),
+ mem.make_string(Some(array_type_name)),
+ ]),
+ )
+ .uncast()
+}
diff --git a/pgdog/src/frontend/client/query_engine/multi_step/subquery/mod.rs b/pgdog/src/frontend/client/query_engine/multi_step/subquery/mod.rs
index 26cf31a64..dfc13b5bb 100644
--- a/pgdog/src/frontend/client/query_engine/multi_step/subquery/mod.rs
+++ b/pgdog/src/frontend/client/query_engine/multi_step/subquery/mod.rs
@@ -14,7 +14,7 @@ mod evaluate;
mod request;
mod rewrite;
-use pgdog_postgres_types::DataType;
+use pgdog_postgres_types::{Array, DataType};
use super::Error;
use crate::{
@@ -55,6 +55,33 @@ pub(crate) struct SubqueryMulti<'ast> {
}
impl<'ast> SubqueryMulti<'ast> {
+ /// Returns true if we can send all columns of `data_rows` as unnest(..) arguments, i.e., if
+ /// - the types of the columns are built into Postgres (e.g., not enums)
+ /// - the values of array columns are 1D arrays that start at index 1 (e.g., not {{1,2},{3,4}}).
+ fn can_send_as_unnest_arguments(
+ data_rows: &[DataRow],
+ row_description: &RowDescription,
+ ) -> bool {
+ row_description
+ .fields
+ .iter()
+ .enumerate()
+ .all(|(idx, field)| {
+ if DataType::array_type_oid(field.type_oid).is_some() {
+ return true;
+ }
+
+ DataType::array_element_oid(field.type_oid).is_some()
+ && data_rows.iter().all(|row| {
+ row.get_raw(idx)
+ .filter(|value| !value.is_null)
+ .is_none_or(|value| {
+ Array::binary_elements(value.data.as_ref()).is_some()
+ })
+ })
+ })
+ }
+
pub(crate) fn new(subqueries: Vec<Subquery>, client_request: &'ast ClientRequest) -> Self {
let ast = client_request
.ast
@@ -190,23 +217,11 @@ impl<'ast> SubqueryMulti<'ast> {
let results =
results?.expect("send_query_string is only None when `is_subquery` is false");
- // We send the rows of subqueries as unnest(..) arrays, so we check the column types before rewriting to ensure that all columns have array types.
+ // We send the rows of subqueries as unnest(..) arrays, so we check that we can send all columns as arrays (`can_send_as_unnest_arguments`) before rewriting.
// EXISTS is the only exception to this, since we only read whether or not the subquery returned rows (exists just needs 1 row to be replaced w/ TRUE)
- //
- // Specifically, we return the error below for the columns of the types that `DataType::array_type_oid` doesn't return an array type for:
- // - array columns (e.g., bigint[]), since 2D arrays need inner arrays of the same length, but the arrays of the rows can have different lengths (e.g., {a,b} and {c}).
- // - columns of types that extensions or users create (e.g., enums) (see below)
- //
- // TODO(jkaczman): Support columns who have enum type e.g., mood (CREATE TYPE mood AS ENUM ('sad', 'happy')) or citext.
- // The RowDescription gives us the OID of the type (e.g., 16385). However, the OID can vary between shards. The good thing is that the name of the type (mood) is the same on all shards. This allows us to first look up the name of the type in pg_type (on the shard that returned the OID)
- // Then, we could send the values as a text[], and cast that array with the name of the type instead of the OID, e.g., $1::pg_catalog._text::mood[].
- // How can the values go into the text[] as-is? In binary format, enum or citext value is the same bytes as the text value (e.g., 'happy' is the 5 bytes h, a, p, p, y).
if !self.is_exists_in_where(subquery)
- && let (_, Some(row_description)) = &results
- && row_description
- .fields
- .iter()
- .any(|field| DataType::array_type_oid(field.type_oid).is_none())
+ && let (data_rows, Some(row_description)) = &results
+ && !Self::can_send_as_unnest_arguments(data_rows, row_description)
{
return Err(Error::Subquery(
SubqueryError::SubqueryReturningUnsupportedColumn,
^ permalink raw [nested|flat] 6+ messages in thread
* Re: [pgdogdev/pgdog] PR #2061: feat: support array columns encountered in subqueries
@ 2026-11-18 20:10 "codecov[bot] (bot) (@codecov[bot])" <noreply+codecov[bot]@github.com>
4 siblings, 0 replies; 6+ messages in thread
From: codecov[bot] (bot) (@codecov[bot]) @ 2026-11-18 20:10 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
## [Codecov](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2061?dropdown=coverage&src=pr&el=h1&utm_me...) Report
:x: Patch coverage is `99.42529%` with `1 line` in your changes missing coverage. Please review.
| [Files with missing lines](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2061?dropdown=coverage&src=pr&el=tree&utm_...) | Patch % | Lines |
|---|---|---|
| [...y\_engine/multi\_step/subquery/clause/from\_clause.rs](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2061?src=pr&el=tree&utm_medium=referral&ut...) | 98.95% | [1 Missing :warning: ](https://app.codecov.io/gh/pgdogdev/pgdog/pull/2061?src=pr&el=tree&utm_medium=referral&ut...) |
:loudspeaker: Thoughts on this report? [Let us know!](https://github.com/codecov/feedback/issues/255)
^ permalink raw [nested|flat] 6+ messages in thread
* Re: [pgdogdev/pgdog] PR #2061: feat: support array columns encountered in subqueries
@ 2026-11-18 20:49 "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
4 siblings, 0 replies; 6+ messages in thread
From: github-actions[bot] (bot) (@github-actions[bot]) @ 2026-11-18 20:49 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
<!-- metal-bench -->
**Metal bench (arm64) passed.** All 520 measures within bounds.
| suite | throughput | cpu/txn | user cycles/txn | user instr/txn | kernel cycles/txn | kernel instr/txn | ctx/txn |
|---|---|---|---|---|---|---|---|
| pooler | | 19.18 (+0.6% ± 2.7%) | 18,013 (+0.5% ± 0.7%) | 31,371 (-0.0% ± 0.2%) | 31,076 (+0.8% ± 4.7%) | 51,141 (+0.1% ± 0.4%) | |
| lb | | 19.64 (-0.1% ± 1.6%) | 18,640 (+0.6% ± 0.8%) | 32,270 (-0.1% ± 0.2%) | 31,468 (-0.6% ± 2.9%) | 52,237 (+0.0% ± 0.4%) | |
| sharding-w2 | | 32.01 (+0.4% ± 2.6%) | 42,772 (-0.0% ± 1.2%) | 94,898 (+0.0% ± 0.02%) | 40,401 (+1.1% ± 5.6%) | 53,626 (-0.0% ± 0.09%) | 1.548 (-0.0% ± 0.3%) |
| sharding-w8 | 246,632 (+0.1% ± 0.7%) | 28.92 (-0.2% ± 0.6%) | 43,634 (-0.4% ± 2.4%) | 50,474 (-0.0% ± 0.1%) | 36,231 (+0.1% ± 4.0%) | 42,371 (+0.0% ± 0.05%) | 0.1722 (+0.9% ± 0.7%) |
<details><summary>Counters per txn</summary>
| suite | frontend stall | l1i refill | backend stall |
|---|---|---|---|
| pooler | 7,731 (+1.6% ± 2.3%) | 2,677 (+0.5% ± 0.8%) | |
| lb | 8,156 (+2.0% ± 2.0%) | 2,734 (+0.6% ± 0.7%) | |
| sharding-w2 | 17,230 (-0.1% ± 2.9%) | 4,096 (-0.2% ± 1.4%) | |
| sharding-w8 | 19,302 (-0.5% ± 4.4%) | 4,014 (-0.2% ± 2.2%) | |
</details>
Microbench: all 507 measures within bounds.
<details><summary>Hot code per txn</summary>
| path | base | head | change |
|---|---|---|---|
| pooler | 113.5 KB | 114.1 KB | **+0.6 KB** |
| sharding-w2 | 167.2 KB | 166.7 KB | **-0.5 KB** |
pooler:
| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::client::query_engine | 20.0 KB | 19.9 KB | -0.1 KB |
| tokio | 18.3 KB | 18.9 KB | +0.6 KB |
| pgdog::backend::pool | 17.9 KB | 17.8 KB | -0.1 KB |
| aws-lc (TLS crypto) | 10.3 KB | 10.2 KB | -0.1 KB |
| rustls (TLS) | 8.3 KB | 8.5 KB | +0.2 KB |
| pgdog::frontend (other) | 7.9 KB | 7.8 KB | -0.1 KB |
| pgdog::backend::server | 7.9 KB | 7.7 KB | -0.2 KB |
| other | 5.9 KB | 5.9 KB | +0.0 KB |
| everything else | 17.0 KB | 17.4 KB | +0.4 KB |
| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 5.6 KB | +0.0 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 5.2 KB | -0.1 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 4.1 KB | +0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 3.4 KB | -0.1 KB |
| `tokio::runtime::task::raw::poll` | 3.3 KB | +0.2 KB |
sharding-w2:
| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::router | 29.1 KB | 29.2 KB | +0.1 KB |
| pgdog::frontend::client::query_engine | 20.9 KB | 21.1 KB | +0.2 KB |
| tokio | 18.7 KB | 18.5 KB | -0.2 KB |
| pgdog::backend::pool | 18.2 KB | 18.2 KB | +0.1 KB |
| aws-lc (TLS crypto) | 12.6 KB | 12.9 KB | +0.4 KB |
| other | 9.9 KB | 9.8 KB | -0.1 KB |
| pgdog::frontend (other) | 8.9 KB | 9.0 KB | +0.1 KB |
| pgdog::backend::server | 7.9 KB | 8.2 KB | +0.4 KB |
| everything else | 41.0 KB | 39.6 KB | -1.4 KB |
| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 6.4 KB | +0.0 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 5.1 KB | -0.1 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 3.9 KB | +0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 3.4 KB | +0.1 KB |
| `pgdog::frontend::router::parser::query::select::<impl pgdog::frontend::router::parser::que` | 3.3 KB | -0.1 KB |
</details>
<details><summary>lb flamegraph vs 07c4e6eb</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-subqueries-array-cols-support/7cd34f4f88ae4cc7...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-subqueries-array-cols-support/7cd34f4f88ae4cc7...).
</details>
<details><summary>microbench flamegraph vs 07c4e6eb</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-subqueries-array-cols-support/7cd34f4f88ae4cc7...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-subqueries-array-cols-support/7cd34f4f88ae4cc7...).
</details>
<details><summary>pooler flamegraph vs 07c4e6eb</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-subqueries-array-cols-support/7cd34f4f88ae4cc7...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-subqueries-array-cols-support/7cd34f4f88ae4cc7...).
</details>
<details><summary>sharding-w2 flamegraph vs 07c4e6eb</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-subqueries-array-cols-support/7cd34f4f88ae4cc7...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-subqueries-array-cols-support/7cd34f4f88ae4cc7...).
</details>
<details><summary>sharding-w8 flamegraph vs 07c4e6eb</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-subqueries-array-cols-support/7cd34f4f88ae4cc7...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results/jk-subqueries-array-cols-support/7cd34f4f88ae4cc7...).
</details>
<details><summary>Logs</summary>
[lb](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/jk-subqueries-array-col...), [microbench](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/jk-subqueries-array-col...), [pooler](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/jk-subqueries-array-col...), [sharding-w2](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/jk-subqueries-array-col...), [sharding-w8](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/jk-subqueries-array-col...), [verdict.json](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results/jk-subqueries-array-col...)
</details>
Compared 7cd34f4f with 07c4e6eb on c8g.metal-24xl (spot).
^ permalink raw [nested|flat] 6+ messages in thread
* Re: [pgdogdev/pgdog] PR #2061: feat: support array columns encountered in subqueries
@ 2026-11-18 21:05 "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
4 siblings, 0 replies; 6+ messages in thread
From: github-actions[bot] (bot) (@github-actions[bot]) @ 2026-11-18 21:05 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
<!-- metal-bench-x86 -->
**Metal bench (x86) passed.** All 520 measures within bounds.
| suite | throughput | cpu/txn | user cycles/txn | user instr/txn | kernel cycles/txn | kernel instr/txn | ctx/txn |
|---|---|---|---|---|---|---|---|
| pooler | | 34.78 (+0.0% ± 0.08%) | 23,067 (+0.5% ± 0.7%) | 32,725 (-0.0% ± 0.01%) | 57,801 (-0.2% ± 0.2%) | 38,554 (-0.1% ± 0.06%) | |
| lb | | 35.78 (+0.1% ± 0.2%) | 24,227 (+0.5% ± 0.7%) | 33,688 (-0.0% ± 0.02%) | 58,770 (-0.0% ± 0.3%) | 39,215 (-0.0% ± 0.1%) | |
| sharding-w2 | | 56.89 (+0.1% ± 0.6%) | 53,089 (+0.1% ± 1.4%) | 98,573 (+0.0% ± 0.01%) | 74,141 (+0.2% ± 0.8%) | 43,065 (-0.0% ± 0.3%) | 1.389 (-0.1% ± 0.2%) |
| sharding-w8 | 146,675 (-0.2% ± 1.4%) | 48.56 (+0.1% ± 1.9%) | 58,148 (+0.1% ± 2.2%) | 54,056 (+0.0% ± 0.1%) | 65,153 (+0.0% ± 1.7%) | 34,248 (-0.0% ± 0.4%) | 0.1669 (-0.4% ± 0.7%) |
<details><summary>Counters per txn</summary>
| suite | frontend stall | l1i refill | backend stall |
|---|---|---|---|
| pooler | 74,240 (-0.2% ± 1.1%) | 3,701 (+0.1% ± 0.3%) | |
| lb | 77,506 (-0.0% ± 0.5%) | 3,818 (-0.0% ± 0.2%) | |
| sharding-w2 | 161,966 (+0.6% ± 2.9%) | 6,160 (-0.1% ± 1.2%) | |
| sharding-w8 | 122,079 (+0.1% ± 2.2%) | 4,950 (+0.3% ± 0.6%) | |
</details>
Microbench: all 507 measures within bounds.
<details><summary>Microbench latency: 1 benches changed more than 3%</summary>
| benchmark | latency | baseline | change | instructions change |
|---|---|---|---|---|
| full_simple::delete_by_key | 10,518 ns | 10,184 ns | +3.28% ± 1.42% | +0.00% ± 0.00% |
</details>
<details><summary>Hot code per txn</summary>
| path | base | head | change |
|---|---|---|---|
| pooler | 110.9 KB | 112.4 KB | **+1.5 KB** |
| sharding-w2 | 168.2 KB | 167.7 KB | **-0.5 KB** |
pooler:
| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::client::query_engine | 21.2 KB | 21.7 KB | +0.4 KB |
| tokio | 17.6 KB | 17.8 KB | +0.2 KB |
| pgdog::backend::pool | 16.9 KB | 17.1 KB | +0.1 KB |
| pgdog::frontend (other) | 8.9 KB | 9.0 KB | +0.1 KB |
| pgdog::backend::server | 7.6 KB | 7.9 KB | +0.3 KB |
| aws-lc (TLS crypto) | 7.9 KB | 7.7 KB | -0.2 KB |
| rustls (TLS) | 7.1 KB | 7.6 KB | +0.4 KB |
| other | 5.9 KB | 6.1 KB | +0.2 KB |
| everything else | 17.8 KB | 17.6 KB | -0.2 KB |
| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 6.9 KB | +0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 6.5 KB | +0.1 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 4.6 KB | +0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 3.8 KB | +0.2 KB |
| `tokio::runtime::task::raw::poll` | 3.6 KB | +0.0 KB |
sharding-w2:
| crate | base | head | change |
|---|---|---|---|
| pgdog::frontend::router | 27.2 KB | 27.4 KB | +0.2 KB |
| pgdog::frontend::client::query_engine | 23.2 KB | 23.9 KB | +0.8 KB |
| tokio | 17.8 KB | 18.4 KB | +0.6 KB |
| pgdog::backend::pool | 18.7 KB | 18.2 KB | -0.4 KB |
| other | 11.2 KB | 10.6 KB | -0.6 KB |
| aws-lc (TLS crypto) | 10.7 KB | 10.1 KB | -0.6 KB |
| jemalloc | 10.1 KB | 9.9 KB | -0.2 KB |
| pgdog::frontend (other) | 10.0 KB | 9.9 KB | -0.1 KB |
| everything else | 39.2 KB | 39.2 KB | +0.0 KB |
| function | head | change |
|---|---|---|
| `pgdog::frontend::client::Client::run::{{closure}}` | 7.4 KB | -0.1 KB |
| `pgdog::frontend::client::query_engine::query::<impl pgdog::frontend::client::query_engine:` | 6.4 KB | +0.1 KB |
| `pgdog::frontend::client::query_engine::QueryEngine::handle::{{closure}}` | 4.6 KB | +0.1 KB |
| `pgdog::frontend::router::parser::query::select::<impl pgdog::frontend::router::parser::que` | 4.1 KB | -0.1 KB |
| `tokio::runtime::task::raw::poll` | 3.9 KB | +0.1 KB |
</details>
<details><summary>lb flamegraph vs 07c4e6eb</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-subqueries-array-cols-support/7cd34f4f88ae...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-subqueries-array-cols-support/7cd34f4f88ae...).
</details>
<details><summary>microbench flamegraph vs 07c4e6eb</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-subqueries-array-cols-support/7cd34f4f88ae...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-subqueries-array-cols-support/7cd34f4f88ae...).
</details>
<details><summary>pooler flamegraph vs 07c4e6eb</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-subqueries-array-cols-support/7cd34f4f88ae...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-subqueries-array-cols-support/7cd34f4f88ae...).
</details>
<details><summary>sharding-w2 flamegraph vs 07c4e6eb</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-subqueries-array-cols-support/7cd34f4f88ae...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-subqueries-array-cols-support/7cd34f4f88ae...).
</details>
<details><summary>sharding-w8 flamegraph vs 07c4e6eb</summary>

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-subqueries-array-cols-support/7cd34f4f88ae...) for hover, zoom and search.

[Open interactive](https://pgdog-metal-bench.s3.amazonaws.com/results-x86/jk-subqueries-array-cols-support/7cd34f4f88ae...).
</details>
<details><summary>Logs</summary>
[lb](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/jk-subqueries-array...), [microbench](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/jk-subqueries-array...), [pooler](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/jk-subqueries-array...), [sharding-w2](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/jk-subqueries-array...), [sharding-w8](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/jk-subqueries-array...), [verdict.json](https://s3.console.aws.amazon.com/s3/object/pgdog-metal-bench?prefix=results-x86/jk-subqueries-array...)
</details>
Compared 7cd34f4f with 07c4e6eb on m7a.metal-48xl (spot).
^ permalink raw [nested|flat] 6+ messages in thread
* Re: [pgdogdev/pgdog] PR #2061: feat: support array columns encountered in subqueries
@ 2026-11-18 22:50 "jkaczman (@jkaczman)" <noreply+jkaczman@github.com>
4 siblings, 0 replies; 6+ messages in thread
From: jkaczman (@jkaczman) @ 2026-11-18 22:50 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
@jkaczman merged this (commit 8cfcf31f6901).
^ permalink raw [nested|flat] 6+ messages in thread
* Re: [pgdogdev/pgdog] PR #2061: feat: support array columns encountered in subqueries
@ 2026-11-18 22:50 "jkaczman (@jkaczman)" <noreply+jkaczman@github.com>
4 siblings, 0 replies; 6+ messages in thread
From: jkaczman (@jkaczman) @ 2026-11-18 22:50 UTC (permalink / raw)
To: pgdogdev/pgdog <noreply+pgdogdev-pgdog@github.com>
@jkaczman closed this.
^ permalink raw [nested|flat] 6+ messages in thread
end of thread, other threads:[~2026-11-18 22:50 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-11-18 17:16 [pgdogdev/pgdog] PR #2061: feat: support array columns encountered in subqueries "jkaczman (@jkaczman)" <noreply+jkaczman@github.com>
2026-11-18 20:10 ` "codecov[bot] (bot) (@codecov[bot])" <noreply+codecov[bot]@github.com>
2026-11-18 20:49 ` "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
2026-11-18 21:05 ` "github-actions[bot] (bot) (@github-actions[bot])" <noreply+github-actions[bot]@github.com>
2026-11-18 22:50 ` "jkaczman (@jkaczman)" <noreply+jkaczman@github.com>
2026-11-18 22:50 ` "jkaczman (@jkaczman)" <noreply+jkaczman@github.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox