pg.ddx.io  pgpartman/pg_partman GitHub issues and pull requests  
help / color / mirror / Atom feed
From: ritojoao (@ritojoao) <noreply+ritojoao@github.com>
To: pgpartman/pg_partman <noreply+pgpartman-pg_partman@github.com>
Subject: [pgpartman/pg_partman] PR #887: Compare partition_data_time() bounds as timestamptz, not text
Date: Tue, 10 Nov 2026 13:47:12 +0000
Message-ID: <gh-pgpartman-pg_partman-887@github.com> (raw)

Fixes #868.

### Problem

`partition_data_time()` casts the control column to text and compares it with the text form of the child table bounds. For a `date` column, `'2002-07-01'` sorts before `'2002-07-01 00:00:00+00'`, so a row on the first day of a month is picked up together with the previous month. The move then fails with:

```
ERROR:  new row for relation "events_p20020601" violates partition constraint
```

The reproduction script in #868 shows it with `partition_data_proc()` on a monthly `date` partition set.

### Fix

Compare the control column directly with the bounds cast to `timestamptz`:

```sql
WHERE col >= '...'::timestamptz AND col < '...'::timestamptz
```

- `date` and `timestamp` columns are compared with PostgreSQL's own cross-type operators, so no text ordering is involved.
- This also covers `timestamp without time zone` (#838), so the special case for it is no longer needed.
- The comparison no longer wraps the column in a cast, so an index on the control column can be used.
- The time decoder path is unchanged apart from the cast on the bounds.

I only changed `sql/functions/partition_data_time.sql`. I left the update script alone, as the other recent fixes did. I'm happy to add it if you prefer.

### Tests

- New test `test/test-time-date-monthly-boundary.sql`: it moves first-of-month rows for a `date` and a `timestamp` control column, with `timezone = 'America/New_York'`. It fails before the change and passes after.
- I ran the top-level tests (`pg_prove test/*.sql`) on PostgreSQL 17. All pass except `test-dump-definition.sql` test 2, which also fails on `development` without this change in my setup.
- I ran the reproduction script from #868 by hand: all three rows end up in the right child tables.

Thank you for looking at this!


inline patch 1: sql/functions/partition_data_time.sql, test/test-time-date-monthly-boundary.sql (+62/-11 lines) download inline-1.patchdiff --git a/sql/functions/partition_data_time.sql b/sql/functions/partition_data_time.sql index 0f72f5c..0aeaf00 100644 --- a/sql/functions/partition_data_time.sql +++ b/sql/functions/partition_data_time.sql @@ -21,7 +21,6 @@ v_async_rowcount int; v_column_list_filtered text; v_column_list_full text; v_control text; -v_control_exact_type text; v_control_type text; v_datetime_string text; v_current_partition_name text; @@ -91,7 +90,7 @@ FROM pg_catalog.pg_tables WHERE schemaname = split_part(p_parent_table, '.', 1)::name AND tablename = split_part(p_parent_table, '.', 2)::name; -SELECT general_type, exact_type INTO v_control_type, v_control_exact_type FROM @extschema@.check_control_type(v_parent_schemaname, v_parent_tablename, v_control); +SELECT general_type INTO v_control_type FROM @extschema@.check_control_type(v_parent_schemaname, v_parent_tablename, v_control); IF v_control_type <> 'time' THEN IF (v_control_type = 'id' AND v_epoch = 'none') OR v_control_type NOT IN ('text', 'id', 'uuid') OR (v_control_type IN ('text', 'uuid') AND v_time_encoder IS NULL) THEN RAISE EXCEPTION 'Cannot run on partition set without time based control column, an epoch flag set with an id column or time_encoder set with text column. Found control: %, epoch: %, time_encoder: %s', v_control_type, v_epoch, v_time_encoder; @@ -315,14 +314,11 @@ FOR i IN 1..p_batch_count LOOP -- Create a variable to use in all scenarios below for handling encoded columns + -- The boundary values below are always compared as timestamptz, never as text. Text comparison + -- puts rows in the wrong child table when the text format of the column differs from timestamptz, + -- for example date values (See Github Issues #838 and #868) IF v_time_decoder IS NULL THEN - IF v_control_exact_type = 'timestamp' THEN - -- Conversion to text of timestamp without timezone cannot be directly compared to timestamptz - -- that is returned by other functions and used below (See Github Issue #838) - v_decoded_col := format('%s::timestamptz::text', v_partition_expression); - ELSE - v_decoded_col := format('%s::text', v_partition_expression); - END IF; + v_decoded_col := v_partition_expression; ELSE v_decoded_col := format('%s(%s::text)', v_time_decoder_safe, v_partition_expression); END IF; @@ -338,7 +334,7 @@ FOR i IN 1..p_batch_count LOOP -- Infinity only needs to be of concern when being removed from the default - v_sql := format('DELETE FROM %1$I.%2$I WHERE %3$s >= %4$L AND %3$s < %5$L RETURNING *' + v_sql := format('DELETE FROM %1$I.%2$I WHERE %3$s >= %4$L::timestamptz AND %3$s < %5$L::timestamptz RETURNING *' , v_source_schemaname , v_source_tablename , v_decoded_col @@ -369,7 +365,7 @@ FOR i IN 1..p_batch_count LOOP v_analyze := @extschema@.create_partition_time(p_parent_table, v_partition_timestamp); v_sql := format('WITH partition_data AS ( - DELETE FROM ONLY %I.%I WHERE %3$s >= %L AND %3$s < %5$L RETURNING *) + DELETE FROM ONLY %I.%I WHERE %3$s >= %L::timestamptz AND %3$s < %5$L::timestamptz RETURNING *) INSERT INTO %6$I.%7$I (%8$s) %9$s SELECT %8$s FROM partition_data' , v_source_schemaname , v_source_tablename diff --git a/test/test-time-date-monthly-boundary.sql b/test/test-time-date-monthly-boundary.sql new file mode 100644 index 0000000..f945d1a --- /dev/null +++ b/test/test-time-date-monthly-boundary.sql @@ -0,0 +1,55 @@ +-- ########## TIME MONTHLY DATE BOUNDARY ########## +-- Other tests: + -- partition_data_time() with a date and a timestamp without time zone control column + -- Rows on the first day of a month must go to that month's child table (See Github Issues #838 and #868) + -- Uses a non-UTC time zone so timestamptz text output has a non-zero offset + +\set ON_ERROR_ROLLBACK 1 +\set ON_ERROR_STOP true + +BEGIN; +SELECT set_config('search_path','partman, public',false); +SET LOCAL timezone = 'America/New_York'; + +SELECT plan(14); +CREATE SCHEMA partman_test; + +CREATE TABLE partman_test.date_taptest_table + (col1 int NOT NULL + , col2 date NOT NULL) + PARTITION BY RANGE (col2); + +CREATE TABLE partman_test.ts_taptest_table + (col1 int NOT NULL + , col2 timestamp NOT NULL) + PARTITION BY RANGE (col2); + +SELECT create_parent('partman_test.date_taptest_table', 'col2', '1 month'); +SELECT create_parent('partman_test.ts_taptest_table', 'col2', '1 month'); + +-- Old values far from now, so they all go to the default table first +INSERT INTO partman_test.date_taptest_table (col1, col2) VALUES + (1, '2002-06-15'), (2, '2002-07-01'), (3, '2002-07-15'), (4, '2002-08-01'); +INSERT INTO partman_test.ts_taptest_table (col1, col2) VALUES + (1, '2002-06-15 12:00:00'), (2, '2002-07-01 00:00:00'), (3, '2002-07-15 12:00:00'), (4, '2002-08-01 00:00:00'); + +SELECT results_eq('SELECT partition_data_time(''partman_test.date_taptest_table'', p_batch_count := 10)::int', ARRAY[4], 'Check that partition_data_time moved all 4 rows of date_taptest_table'); + +SELECT has_table('partman_test', 'date_taptest_table_p20020601', 'Check date_taptest_table_p20020601 exists'); +SELECT has_table('partman_test', 'date_taptest_table_p20020701', 'Check date_taptest_table_p20020701 exists'); +SELECT has_table('partman_test', 'date_taptest_table_p20020801', 'Check date_taptest_table_p20020801 exists'); +SELECT results_eq('SELECT col1 FROM partman_test.date_taptest_table_p20020601 ORDER BY col1', ARRAY[1], 'Check rows in date_taptest_table_p20020601'); +SELECT results_eq('SELECT col1 FROM partman_test.date_taptest_table_p20020701 ORDER BY col1', ARRAY[2,3], 'Check rows in date_taptest_table_p20020701'); +SELECT results_eq('SELECT col1 FROM partman_test.date_taptest_table_p20020801 ORDER BY col1', ARRAY[4], 'Check rows in date_taptest_table_p20020801'); +SELECT is_empty('SELECT * FROM partman_test.date_taptest_table_default', 'Check that date_taptest_table_default is empty'); + +SELECT results_eq('SELECT partition_data_time(''partman_test.ts_taptest_table'', p_batch_count := 10)::int', ARRAY[4], 'Check that partition_data_time moved all 4 rows of ts_taptest_table'); + +SELECT has_table('partman_test', 'ts_taptest_table_p20020701', 'Check ts_taptest_table_p20020701 exists'); +SELECT results_eq('SELECT col1 FROM partman_test.ts_taptest_table_p20020601 ORDER BY col1', ARRAY[1], 'Check rows in ts_taptest_table_p20020601'); +SELECT results_eq('SELECT col1 FROM partman_test.ts_taptest_table_p20020701 ORDER BY col1', ARRAY[2,3], 'Check rows in ts_taptest_table_p20020701'); +SELECT results_eq('SELECT col1 FROM partman_test.ts_taptest_table_p20020801 ORDER BY col1', ARRAY[4], 'Check rows in ts_taptest_table_p20020801'); +SELECT is_empty('SELECT * FROM partman_test.ts_taptest_table_default', 'Check that ts_taptest_table_default is empty'); + +SELECT * FROM finish(); +ROLLBACK;

view thread (5+ messages)  latest in thread

Message-ID: <gh-pgpartman-pg_partman-887@github.com>
Permalink:  ../gh-pgpartman-pg_partman-887@github.com/

 ·  · 


This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox