public inbox for [email protected]help / color / mirror / Atom feed
Dump statistic issue with index on expressions 3+ messages / 2 participants [nested] [flat]
* Dump statistic issue with index on expressions @ 2026-02-25 16:37 Maksim.Melnikov <[email protected]> 0 siblings, 1 reply; 3+ messages in thread From: Maksim.Melnikov @ 2026-02-25 16:37 UTC (permalink / raw) To: [email protected] Hi hackers. There is an issue on new feature dump statistics related to index processing. In case when table has more then one index and if one of them is index on expressions we can get error like this: pg_dump --verbose --statistics-only > /dev/null ... pg_dump: reading subscriptions pg_dump: reading subscription membership of tables pg_dump: reading dependency data pg_dump: saving encoding = UTF8 pg_dump: saving "standard_conforming_strings = on" pg_dump: saving "search_path = " pg_dump: error: could not find index attname "source_system" For clarity, schema ddl attached CREATE TABLE test_table_stats ( id uuid NOT NULL, body jsonb, source_system character varying, source_id character varying, model_name character varying NOT NULL ); CREATE INDEX test_table_stats_source_system_text ON test_table_stats USING btree (upper((source_system)::text)); CREATE UNIQUE INDEX test_table_stats_json_system_un ON test_table_stats USING btree (source_system, source_id, model_name); When pg_dump sequentially process indexes in case when index is processed after index on expression, it can use index attrs names of previously processed index. I've attached simple patch to fix it. Best regards Melnikov Maksim Attachments: [text/x-patch] 0001-Dump-statistic-issue-with-index-on-expressions.patch (1.5K, 2-0001-Dump-statistic-issue-with-index-on-expressions.patch) download | inline diff: From 7af2b54d114ee2cb6bf55900e3f93b670bd29422 Mon Sep 17 00:00:00 2001 From: Maksim Melnikov <[email protected]> Date: Wed, 25 Feb 2026 15:43:17 +0300 Subject: [PATCH] Dump statistic issue with index on expressions. There is an issue on new feature dump statistics related to index processing. In case when table has more then one index and if one of them is index on expressions we can get dump error: could not find index attname "att_name". When pg_dump sequentially process indexes in case when index is processed after index on expression, it can use index attrs names of previously processed index. --- src/bin/pg_dump/pg_dump.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c index 450cec285b3..7bdc2a027d9 100644 --- a/src/bin/pg_dump/pg_dump.c +++ b/src/bin/pg_dump/pg_dump.c @@ -8070,8 +8070,6 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables) { Oid indrelid = atooid(PQgetvalue(res, j, i_indrelid)); TableInfo *tbinfo = NULL; - char **indAttNames = NULL; - int nindAttNames = 0; int numinds; /* Count rows for this table */ @@ -8105,6 +8103,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables) { char contype; char indexkind; + char **indAttNames = NULL; + int nindAttNames = 0; RelStatsInfo *relstats; int32 relpages = atoi(PQgetvalue(res, j, i_relpages)); int32 relallvisible = atoi(PQgetvalue(res, j, i_relallvisible)); -- 2.43.0 ^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Dump statistic issue with index on expressions @ 2026-05-20 09:51 Maksim.Melnikov <[email protected]> parent: Maksim.Melnikov <[email protected]> 0 siblings, 1 reply; 3+ messages in thread From: Maksim.Melnikov @ 2026-05-20 09:51 UTC (permalink / raw) To: Alexander Korotkov <[email protected]>; +Cc: [email protected] Alexander, hello! On 5/11/26 02:42, Alexander Korotkov wrote: > Hi, Maksim! > > On Wed, Feb 25, 2026 at 6:37 PM Maksim.Melnikov > <[email protected]> wrote: >> There is an issue on new feature dump statistics related to index >> processing. >> In case when table has more then one index and if one of them is index >> on expressions >> we can get error like this: >> >> pg_dump --verbose --statistics-only > /dev/null >> ... >> pg_dump: reading subscriptions >> pg_dump: reading subscription membership of tables >> pg_dump: reading dependency data >> pg_dump: saving encoding = UTF8 >> pg_dump: saving "standard_conforming_strings = on" >> pg_dump: saving "search_path = " >> pg_dump: error: could not find index attname "source_system" >> >> For clarity, schema ddl attached >> >> CREATE TABLE test_table_stats ( >> id uuid NOT NULL, >> body jsonb, >> source_system character varying, >> source_id character varying, >> model_name character varying NOT NULL >> ); >> >> CREATE INDEX test_table_stats_source_system_text ON test_table_stats >> USING btree (upper((source_system)::text)); >> CREATE UNIQUE INDEX test_table_stats_json_system_un ON test_table_stats >> USING btree (source_system, source_id, model_name); >> >> When pg_dump sequentially process indexes in case when index is >> processed after index on expression, >> it can use index attrs names of previously processed index. I've >> attached simple patch to fix it. > I see this is a bug indeed: an index with no expression can get its > indAttNames and nindAttNames from the previous index. But I didn't > manage to reproduce your case. dumpRelationStats_dumper() only > iterates indexes with pg_stats entry, and those are indexes with > expressions. Could you give more details on how did you reproduce > user-facing error? Which particular git commit did you use? How did > you fill the database step by step? > > ------ > Regards, > Alexander Korotkov > Supabase > > Sorry for delay, I've tried to reproduce this issue for some time and detect that it isn't reproducable on vanilla postgresql. It seems, that our fork generate more records for pg_statistics unlike vanilla, that generate records only for indexes with expressions, more details can be found here in src/backend/commands/analyze.c static void do_analyze_rel(Relation onerel, VacuumParams *params, List *va_cols, AcquireSampleRowsFunc acquirefunc, BlockNumber relpages, bool inh, bool in_outer_xact, int elevel) { ...... thisdata->tupleFract = 1.0; /* fix later if partial */ if (indexInfo->ii_Expressions != NIL && va_cols == NIL) { ListCell *indexpr_item = list_head(indexInfo->ii_Expressions); thisdata->vacattrstats = (VacAttrStats **) palloc(indexInfo->ii_NumIndexAttrs * sizeof(VacAttrStats *)); ..... thisdata->attr_cnt = tcnt; Anyway, it seems code, reported before, isn't ideal and prone to bugs. In my opinion better fix it. Hope it will be helpful. Best regards, Maksim Melnikov ^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Dump statistic issue with index on expressions @ 2026-05-30 19:36 Alexander Korotkov <[email protected]> parent: Maksim.Melnikov <[email protected]> 0 siblings, 0 replies; 3+ messages in thread From: Alexander Korotkov @ 2026-05-30 19:36 UTC (permalink / raw) To: Maksim.Melnikov <[email protected]>; +Cc: [email protected] Hi, Maksim! On Wed, May 20, 2026 at 12:52 PM Maksim.Melnikov <[email protected]> wrote: > On 5/11/26 02:42, Alexander Korotkov wrote: > > On Wed, Feb 25, 2026 at 6:37 PM Maksim.Melnikov > > <[email protected]> wrote: > >> There is an issue on new feature dump statistics related to index > >> processing. > >> In case when table has more then one index and if one of them is index > >> on expressions > >> we can get error like this: > >> > >> pg_dump --verbose --statistics-only > /dev/null > >> ... > >> pg_dump: reading subscriptions > >> pg_dump: reading subscription membership of tables > >> pg_dump: reading dependency data > >> pg_dump: saving encoding = UTF8 > >> pg_dump: saving "standard_conforming_strings = on" > >> pg_dump: saving "search_path = " > >> pg_dump: error: could not find index attname "source_system" > >> > >> For clarity, schema ddl attached > >> > >> CREATE TABLE test_table_stats ( > >> id uuid NOT NULL, > >> body jsonb, > >> source_system character varying, > >> source_id character varying, > >> model_name character varying NOT NULL > >> ); > >> > >> CREATE INDEX test_table_stats_source_system_text ON test_table_stats > >> USING btree (upper((source_system)::text)); > >> CREATE UNIQUE INDEX test_table_stats_json_system_un ON test_table_stats > >> USING btree (source_system, source_id, model_name); > >> > >> When pg_dump sequentially process indexes in case when index is > >> processed after index on expression, > >> it can use index attrs names of previously processed index. I've > >> attached simple patch to fix it. > > I see this is a bug indeed: an index with no expression can get its > > indAttNames and nindAttNames from the previous index. But I didn't > > manage to reproduce your case. dumpRelationStats_dumper() only > > iterates indexes with pg_stats entry, and those are indexes with > > expressions. Could you give more details on how did you reproduce > > user-facing error? Which particular git commit did you use? How did > > you fill the database step by step? > > > > ------ > > Regards, > > Alexander Korotkov > > Supabase > > > > > Sorry for delay, I've tried to reproduce this issue for some time and > detect that it isn't reproducable on vanilla postgresql. > It seems, that our fork generate more records for pg_statistics unlike > vanilla, that generate records only for indexes with expressions, > more details can be found here in src/backend/commands/analyze.c > > static void > do_analyze_rel(Relation onerel, VacuumParams *params, > List *va_cols, AcquireSampleRowsFunc acquirefunc, > BlockNumber relpages, bool inh, bool in_outer_xact, > int elevel) > { > ...... > thisdata->tupleFract = 1.0; /* fix later if partial */ > if (indexInfo->ii_Expressions != NIL && va_cols == NIL) > { > ListCell *indexpr_item = > list_head(indexInfo->ii_Expressions); > > thisdata->vacattrstats = (VacAttrStats **) > palloc(indexInfo->ii_NumIndexAttrs * > sizeof(VacAttrStats *)); > ..... > thisdata->attr_cnt = tcnt; > > > Anyway, it seems code, reported before, isn't ideal and prone to bugs. > In my opinion better fix it. > Hope it will be helpful. Thank you. Now this is clear. No user-facing error, but an internal inconsistency. I'm going to push (and backpatch) this after release freeze is lifted. ------ Regards, Alexander Korotkov Supabase ^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2026-05-30 19:36 UTC | newest] Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-02-25 16:37 Dump statistic issue with index on expressions Maksim.Melnikov <[email protected]> 2026-05-20 09:51 ` Maksim.Melnikov <[email protected]> 2026-05-30 19:36 ` Alexander Korotkov <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox