pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Fix hstore hash function for empty hstores upgraded from 8.4. 6+ messages / 1 participants [nested] [flat]
* pgsql: Fix hstore hash function for empty hstores upgraded from 8.4. @ 2018-11-24 21:21 Andrew Gierth <rhodiumtoad@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Andrew Gierth @ 2018-11-24 21:21 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix hstore hash function for empty hstores upgraded from 8.4. Hstore data generated on pg 8.4 and pg_upgraded to current versions remains in its original on-disk format unless modified. The same goes for values generated by the addon hstore-new module on pre-9.0 versions. (The hstoreUpgrade function converts old values on the fly when read in, but the on-disk value is not modified by this.) Since old-format empty hstores (and hstore-new hstores) have representations compatible with the new format, hstoreUpgrade thought it could get away without modifying such values; but this breaks hstore_hash (and the new hstore_hash_extended) which assumes bit-perfect matching between semantically identical hstore values. Only one bit actually differs (the "new version" flag in the count field) but that of course is enough to break the hash. Fix by making hstoreUpgrade unconditionally convert all old values to new format. Backpatch all the way, even though this changes a hash value in some cases, because in those cases the hash value is already failing - for example, a hash join between old- and new-format empty hstores will be failing to match, or a hash index on an hstore column containing an old-format empty value will be failing to find the value since it will be searching for a hash derived from a new-format datum. (There are no known field reports of this happening, probably because hashing of hstores has only been useful in limited circumstances and there probably isn't much upgraded data being used this way.) Per concerns arising from discussion of commit eb6f29141be. Original bug is my fault. Discussion: https://postgr.es/m/60b1fd3b-7332-40f0-7e7f-f2f04f777747%402ndquadrant.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/d5890f49da6a77b1325a3f5822c6b092a2cd41ae Modified Files -------------- contrib/hstore/hstore_compat.c | 47 +++++++++++++++++------------------------- 1 file changed, 19 insertions(+), 28 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix hstore hash function for empty hstores upgraded from 8.4. @ 2018-11-24 21:21 Andrew Gierth <rhodiumtoad@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Andrew Gierth @ 2018-11-24 21:21 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix hstore hash function for empty hstores upgraded from 8.4. Hstore data generated on pg 8.4 and pg_upgraded to current versions remains in its original on-disk format unless modified. The same goes for values generated by the addon hstore-new module on pre-9.0 versions. (The hstoreUpgrade function converts old values on the fly when read in, but the on-disk value is not modified by this.) Since old-format empty hstores (and hstore-new hstores) have representations compatible with the new format, hstoreUpgrade thought it could get away without modifying such values; but this breaks hstore_hash (and the new hstore_hash_extended) which assumes bit-perfect matching between semantically identical hstore values. Only one bit actually differs (the "new version" flag in the count field) but that of course is enough to break the hash. Fix by making hstoreUpgrade unconditionally convert all old values to new format. Backpatch all the way, even though this changes a hash value in some cases, because in those cases the hash value is already failing - for example, a hash join between old- and new-format empty hstores will be failing to match, or a hash index on an hstore column containing an old-format empty value will be failing to find the value since it will be searching for a hash derived from a new-format datum. (There are no known field reports of this happening, probably because hashing of hstores has only been useful in limited circumstances and there probably isn't much upgraded data being used this way.) Per concerns arising from discussion of commit eb6f29141be. Original bug is my fault. Discussion: https://postgr.es/m/60b1fd3b-7332-40f0-7e7f-f2f04f777747%402ndquadrant.com Branch ------ REL_11_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/02e669c0f7ddbdc4f1ca74bfa5ccfcfca782f144 Modified Files -------------- contrib/hstore/hstore_compat.c | 47 +++++++++++++++++------------------------- 1 file changed, 19 insertions(+), 28 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix hstore hash function for empty hstores upgraded from 8.4. @ 2018-11-24 21:21 Andrew Gierth <rhodiumtoad@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Andrew Gierth @ 2018-11-24 21:21 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix hstore hash function for empty hstores upgraded from 8.4. Hstore data generated on pg 8.4 and pg_upgraded to current versions remains in its original on-disk format unless modified. The same goes for values generated by the addon hstore-new module on pre-9.0 versions. (The hstoreUpgrade function converts old values on the fly when read in, but the on-disk value is not modified by this.) Since old-format empty hstores (and hstore-new hstores) have representations compatible with the new format, hstoreUpgrade thought it could get away without modifying such values; but this breaks hstore_hash (and the new hstore_hash_extended) which assumes bit-perfect matching between semantically identical hstore values. Only one bit actually differs (the "new version" flag in the count field) but that of course is enough to break the hash. Fix by making hstoreUpgrade unconditionally convert all old values to new format. Backpatch all the way, even though this changes a hash value in some cases, because in those cases the hash value is already failing - for example, a hash join between old- and new-format empty hstores will be failing to match, or a hash index on an hstore column containing an old-format empty value will be failing to find the value since it will be searching for a hash derived from a new-format datum. (There are no known field reports of this happening, probably because hashing of hstores has only been useful in limited circumstances and there probably isn't much upgraded data being used this way.) Per concerns arising from discussion of commit eb6f29141be. Original bug is my fault. Discussion: https://postgr.es/m/60b1fd3b-7332-40f0-7e7f-f2f04f777747%402ndquadrant.com Branch ------ REL_10_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/bcbb682786a3a263d2201c105d52ff51fc23d98e Modified Files -------------- contrib/hstore/hstore_compat.c | 47 +++++++++++++++++------------------------- 1 file changed, 19 insertions(+), 28 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix hstore hash function for empty hstores upgraded from 8.4. @ 2018-11-24 21:21 Andrew Gierth <rhodiumtoad@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Andrew Gierth @ 2018-11-24 21:21 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix hstore hash function for empty hstores upgraded from 8.4. Hstore data generated on pg 8.4 and pg_upgraded to current versions remains in its original on-disk format unless modified. The same goes for values generated by the addon hstore-new module on pre-9.0 versions. (The hstoreUpgrade function converts old values on the fly when read in, but the on-disk value is not modified by this.) Since old-format empty hstores (and hstore-new hstores) have representations compatible with the new format, hstoreUpgrade thought it could get away without modifying such values; but this breaks hstore_hash (and the new hstore_hash_extended) which assumes bit-perfect matching between semantically identical hstore values. Only one bit actually differs (the "new version" flag in the count field) but that of course is enough to break the hash. Fix by making hstoreUpgrade unconditionally convert all old values to new format. Backpatch all the way, even though this changes a hash value in some cases, because in those cases the hash value is already failing - for example, a hash join between old- and new-format empty hstores will be failing to match, or a hash index on an hstore column containing an old-format empty value will be failing to find the value since it will be searching for a hash derived from a new-format datum. (There are no known field reports of this happening, probably because hashing of hstores has only been useful in limited circumstances and there probably isn't much upgraded data being used this way.) Per concerns arising from discussion of commit eb6f29141be. Original bug is my fault. Discussion: https://postgr.es/m/60b1fd3b-7332-40f0-7e7f-f2f04f777747%402ndquadrant.com Branch ------ REL9_6_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/239abfff12a41a0611d24e2c7b67895a7359e6bf Modified Files -------------- contrib/hstore/hstore_compat.c | 47 +++++++++++++++++------------------------- 1 file changed, 19 insertions(+), 28 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix hstore hash function for empty hstores upgraded from 8.4. @ 2018-11-24 21:21 Andrew Gierth <rhodiumtoad@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Andrew Gierth @ 2018-11-24 21:21 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix hstore hash function for empty hstores upgraded from 8.4. Hstore data generated on pg 8.4 and pg_upgraded to current versions remains in its original on-disk format unless modified. The same goes for values generated by the addon hstore-new module on pre-9.0 versions. (The hstoreUpgrade function converts old values on the fly when read in, but the on-disk value is not modified by this.) Since old-format empty hstores (and hstore-new hstores) have representations compatible with the new format, hstoreUpgrade thought it could get away without modifying such values; but this breaks hstore_hash (and the new hstore_hash_extended) which assumes bit-perfect matching between semantically identical hstore values. Only one bit actually differs (the "new version" flag in the count field) but that of course is enough to break the hash. Fix by making hstoreUpgrade unconditionally convert all old values to new format. Backpatch all the way, even though this changes a hash value in some cases, because in those cases the hash value is already failing - for example, a hash join between old- and new-format empty hstores will be failing to match, or a hash index on an hstore column containing an old-format empty value will be failing to find the value since it will be searching for a hash derived from a new-format datum. (There are no known field reports of this happening, probably because hashing of hstores has only been useful in limited circumstances and there probably isn't much upgraded data being used this way.) Per concerns arising from discussion of commit eb6f29141be. Original bug is my fault. Discussion: https://postgr.es/m/60b1fd3b-7332-40f0-7e7f-f2f04f777747%402ndquadrant.com Branch ------ REL9_5_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/8087788f6ade084a6fd696acfa1480890a26b392 Modified Files -------------- contrib/hstore/hstore_compat.c | 47 +++++++++++++++++------------------------- 1 file changed, 19 insertions(+), 28 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix hstore hash function for empty hstores upgraded from 8.4. @ 2018-11-24 21:21 Andrew Gierth <rhodiumtoad@postgresql.org> 0 siblings, 0 replies; 6+ messages in thread From: Andrew Gierth @ 2018-11-24 21:21 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix hstore hash function for empty hstores upgraded from 8.4. Hstore data generated on pg 8.4 and pg_upgraded to current versions remains in its original on-disk format unless modified. The same goes for values generated by the addon hstore-new module on pre-9.0 versions. (The hstoreUpgrade function converts old values on the fly when read in, but the on-disk value is not modified by this.) Since old-format empty hstores (and hstore-new hstores) have representations compatible with the new format, hstoreUpgrade thought it could get away without modifying such values; but this breaks hstore_hash (and the new hstore_hash_extended) which assumes bit-perfect matching between semantically identical hstore values. Only one bit actually differs (the "new version" flag in the count field) but that of course is enough to break the hash. Fix by making hstoreUpgrade unconditionally convert all old values to new format. Backpatch all the way, even though this changes a hash value in some cases, because in those cases the hash value is already failing - for example, a hash join between old- and new-format empty hstores will be failing to match, or a hash index on an hstore column containing an old-format empty value will be failing to find the value since it will be searching for a hash derived from a new-format datum. (There are no known field reports of this happening, probably because hashing of hstores has only been useful in limited circumstances and there probably isn't much upgraded data being used this way.) Per concerns arising from discussion of commit eb6f29141be. Original bug is my fault. Discussion: https://postgr.es/m/60b1fd3b-7332-40f0-7e7f-f2f04f777747%402ndquadrant.com Branch ------ REL9_4_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/e5a6ae97effeca5a80940667e5bffb8672cc7571 Modified Files -------------- contrib/hstore/hstore_compat.c | 47 +++++++++++++++++------------------------- 1 file changed, 19 insertions(+), 28 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2018-11-24 21:21 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2018-11-24 21:21 pgsql: Fix hstore hash function for empty hstores upgraded from 8.4. Andrew Gierth <rhodiumtoad@postgresql.org> 2018-11-24 21:21 pgsql: Fix hstore hash function for empty hstores upgraded from 8.4. Andrew Gierth <rhodiumtoad@postgresql.org> 2018-11-24 21:21 pgsql: Fix hstore hash function for empty hstores upgraded from 8.4. Andrew Gierth <rhodiumtoad@postgresql.org> 2018-11-24 21:21 pgsql: Fix hstore hash function for empty hstores upgraded from 8.4. Andrew Gierth <rhodiumtoad@postgresql.org> 2018-11-24 21:21 pgsql: Fix hstore hash function for empty hstores upgraded from 8.4. Andrew Gierth <rhodiumtoad@postgresql.org> 2018-11-24 21:21 pgsql: Fix hstore hash function for empty hstores upgraded from 8.4. Andrew Gierth <rhodiumtoad@postgresql.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox