pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Fix float4/float8 hash functions to produce uniform results for
7+ messages / 1 participants
[nested] [flat]

* pgsql: Fix float4/float8 hash functions to produce uniform results for
@ 2021-09-02 21:25 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 7+ messages in thread

From: Tom Lane @ 2021-09-02 21:25 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix float4/float8 hash functions to produce uniform results for NaNs.

The IEEE 754 standard allows a wide variety of bit patterns for NaNs,
of which at least two ("NaN" and "-NaN") are pretty easy to produce
from SQL on most machines.  This is problematic because our btree
comparison functions deem all NaNs to be equal, but our float hash
functions know nothing about NaNs and will happily produce varying
hash codes for them.  That causes unexpected results from queries
that hash a column containing different NaN values.  It could also
produce unexpected lookup failures when using a hash index on a
float column, i.e. "WHERE x = 'NaN'" will not find all the rows
it should.

To fix, special-case NaN in the float hash functions, not too much
unlike the existing special case that forces zero and minus zero
to hash the same.  I arranged for the most vanilla sort of NaN
(that coming from the C99 NAN constant) to still have the same
hash code as before, to reduce the risk to existing hash indexes.

I dithered about whether to back-patch this into stable branches,
but ultimately decided to do so.  It's a clear improvement for
queries that hash internally.  If there is anybody who has -NaN
in a hash index, they'd be well advised to re-index after applying
this patch ... but the misbehavior if they don't will not be much
worse than the misbehavior they had before.

Per bug #17172 from Ma Liangzhu.

Discussion: https://postgr.es/m/17172-7505bea9e04e230f@postgresql.org

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/ce773f230d9b5bb2e0dd23fec4e5462fd99487fe

Modified Files
--------------
src/backend/access/hash/hashfunc.c      | 21 +++++++++++++++++++++
src/test/regress/expected/hash_func.out | 33 +++++++++++++++++++++++++++++++++
src/test/regress/sql/hash_func.sql      |  9 +++++++++
3 files changed, 63 insertions(+)



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* pgsql: Fix float4/float8 hash functions to produce uniform results for
@ 2021-09-02 21:25 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 7+ messages in thread

From: Tom Lane @ 2021-09-02 21:25 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix float4/float8 hash functions to produce uniform results for NaNs.

The IEEE 754 standard allows a wide variety of bit patterns for NaNs,
of which at least two ("NaN" and "-NaN") are pretty easy to produce
from SQL on most machines.  This is problematic because our btree
comparison functions deem all NaNs to be equal, but our float hash
functions know nothing about NaNs and will happily produce varying
hash codes for them.  That causes unexpected results from queries
that hash a column containing different NaN values.  It could also
produce unexpected lookup failures when using a hash index on a
float column, i.e. "WHERE x = 'NaN'" will not find all the rows
it should.

To fix, special-case NaN in the float hash functions, not too much
unlike the existing special case that forces zero and minus zero
to hash the same.  I arranged for the most vanilla sort of NaN
(that coming from the C99 NAN constant) to still have the same
hash code as before, to reduce the risk to existing hash indexes.

I dithered about whether to back-patch this into stable branches,
but ultimately decided to do so.  It's a clear improvement for
queries that hash internally.  If there is anybody who has -NaN
in a hash index, they'd be well advised to re-index after applying
this patch ... but the misbehavior if they don't will not be much
worse than the misbehavior they had before.

Per bug #17172 from Ma Liangzhu.

Discussion: https://postgr.es/m/17172-7505bea9e04e230f@postgresql.org

Branch
------
REL_14_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/23c6bc581dc90d08421a13494f135681504ee4e6

Modified Files
--------------
src/backend/access/hash/hashfunc.c      | 21 +++++++++++++++++++++
src/test/regress/expected/hash_func.out | 33 +++++++++++++++++++++++++++++++++
src/test/regress/sql/hash_func.sql      |  9 +++++++++
3 files changed, 63 insertions(+)



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* pgsql: Fix float4/float8 hash functions to produce uniform results for
@ 2021-09-02 21:25 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 7+ messages in thread

From: Tom Lane @ 2021-09-02 21:25 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix float4/float8 hash functions to produce uniform results for NaNs.

The IEEE 754 standard allows a wide variety of bit patterns for NaNs,
of which at least two ("NaN" and "-NaN") are pretty easy to produce
from SQL on most machines.  This is problematic because our btree
comparison functions deem all NaNs to be equal, but our float hash
functions know nothing about NaNs and will happily produce varying
hash codes for them.  That causes unexpected results from queries
that hash a column containing different NaN values.  It could also
produce unexpected lookup failures when using a hash index on a
float column, i.e. "WHERE x = 'NaN'" will not find all the rows
it should.

To fix, special-case NaN in the float hash functions, not too much
unlike the existing special case that forces zero and minus zero
to hash the same.  I arranged for the most vanilla sort of NaN
(that coming from the C99 NAN constant) to still have the same
hash code as before, to reduce the risk to existing hash indexes.

I dithered about whether to back-patch this into stable branches,
but ultimately decided to do so.  It's a clear improvement for
queries that hash internally.  If there is anybody who has -NaN
in a hash index, they'd be well advised to re-index after applying
this patch ... but the misbehavior if they don't will not be much
worse than the misbehavior they had before.

Per bug #17172 from Ma Liangzhu.

Discussion: https://postgr.es/m/17172-7505bea9e04e230f@postgresql.org

Branch
------
REL_13_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/be2beadaff1ad87792efa58273698898d84afd4e

Modified Files
--------------
src/backend/access/hash/hashfunc.c      | 21 +++++++++++++++++++++
src/test/regress/expected/hash_func.out | 33 +++++++++++++++++++++++++++++++++
src/test/regress/sql/hash_func.sql      |  9 +++++++++
3 files changed, 63 insertions(+)



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* pgsql: Fix float4/float8 hash functions to produce uniform results for
@ 2021-09-02 21:25 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 7+ messages in thread

From: Tom Lane @ 2021-09-02 21:25 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix float4/float8 hash functions to produce uniform results for NaNs.

The IEEE 754 standard allows a wide variety of bit patterns for NaNs,
of which at least two ("NaN" and "-NaN") are pretty easy to produce
from SQL on most machines.  This is problematic because our btree
comparison functions deem all NaNs to be equal, but our float hash
functions know nothing about NaNs and will happily produce varying
hash codes for them.  That causes unexpected results from queries
that hash a column containing different NaN values.  It could also
produce unexpected lookup failures when using a hash index on a
float column, i.e. "WHERE x = 'NaN'" will not find all the rows
it should.

To fix, special-case NaN in the float hash functions, not too much
unlike the existing special case that forces zero and minus zero
to hash the same.  I arranged for the most vanilla sort of NaN
(that coming from the C99 NAN constant) to still have the same
hash code as before, to reduce the risk to existing hash indexes.

I dithered about whether to back-patch this into stable branches,
but ultimately decided to do so.  It's a clear improvement for
queries that hash internally.  If there is anybody who has -NaN
in a hash index, they'd be well advised to re-index after applying
this patch ... but the misbehavior if they don't will not be much
worse than the misbehavior they had before.

Per bug #17172 from Ma Liangzhu.

Discussion: https://postgr.es/m/17172-7505bea9e04e230f@postgresql.org

Branch
------
REL_12_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/a3bf136732b13cc3b8f143378e645263ea1aba5e

Modified Files
--------------
src/backend/access/hash/hashfunc.c      | 21 +++++++++++++++++++++
src/test/regress/expected/hash_func.out | 33 +++++++++++++++++++++++++++++++++
src/test/regress/sql/hash_func.sql      |  9 +++++++++
3 files changed, 63 insertions(+)



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* pgsql: Fix float4/float8 hash functions to produce uniform results for
@ 2021-09-02 21:25 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 7+ messages in thread

From: Tom Lane @ 2021-09-02 21:25 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix float4/float8 hash functions to produce uniform results for NaNs.

The IEEE 754 standard allows a wide variety of bit patterns for NaNs,
of which at least two ("NaN" and "-NaN") are pretty easy to produce
from SQL on most machines.  This is problematic because our btree
comparison functions deem all NaNs to be equal, but our float hash
functions know nothing about NaNs and will happily produce varying
hash codes for them.  That causes unexpected results from queries
that hash a column containing different NaN values.  It could also
produce unexpected lookup failures when using a hash index on a
float column, i.e. "WHERE x = 'NaN'" will not find all the rows
it should.

To fix, special-case NaN in the float hash functions, not too much
unlike the existing special case that forces zero and minus zero
to hash the same.  I arranged for the most vanilla sort of NaN
(that coming from the C99 NAN constant) to still have the same
hash code as before, to reduce the risk to existing hash indexes.

I dithered about whether to back-patch this into stable branches,
but ultimately decided to do so.  It's a clear improvement for
queries that hash internally.  If there is anybody who has -NaN
in a hash index, they'd be well advised to re-index after applying
this patch ... but the misbehavior if they don't will not be much
worse than the misbehavior they had before.

Per bug #17172 from Ma Liangzhu.

Discussion: https://postgr.es/m/17172-7505bea9e04e230f@postgresql.org

Branch
------
REL_11_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/ad66373ead6db1b54cfb0af5cda6c24e4cfcaa6b

Modified Files
--------------
src/backend/access/hash/hashfunc.c      | 25 +++++++++++++++++++++++++
src/test/regress/expected/hash_func.out | 33 +++++++++++++++++++++++++++++++++
src/test/regress/sql/hash_func.sql      |  9 +++++++++
3 files changed, 67 insertions(+)



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* pgsql: Fix float4/float8 hash functions to produce uniform results for
@ 2021-09-02 21:25 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 7+ messages in thread

From: Tom Lane @ 2021-09-02 21:25 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix float4/float8 hash functions to produce uniform results for NaNs.

The IEEE 754 standard allows a wide variety of bit patterns for NaNs,
of which at least two ("NaN" and "-NaN") are pretty easy to produce
from SQL on most machines.  This is problematic because our btree
comparison functions deem all NaNs to be equal, but our float hash
functions know nothing about NaNs and will happily produce varying
hash codes for them.  That causes unexpected results from queries
that hash a column containing different NaN values.  It could also
produce unexpected lookup failures when using a hash index on a
float column, i.e. "WHERE x = 'NaN'" will not find all the rows
it should.

To fix, special-case NaN in the float hash functions, not too much
unlike the existing special case that forces zero and minus zero
to hash the same.  I arranged for the most vanilla sort of NaN
(that coming from the C99 NAN constant) to still have the same
hash code as before, to reduce the risk to existing hash indexes.

I dithered about whether to back-patch this into stable branches,
but ultimately decided to do so.  It's a clear improvement for
queries that hash internally.  If there is anybody who has -NaN
in a hash index, they'd be well advised to re-index after applying
this patch ... but the misbehavior if they don't will not be much
worse than the misbehavior they had before.

Per bug #17172 from Ma Liangzhu.

Discussion: https://postgr.es/m/17172-7505bea9e04e230f@postgresql.org

Branch
------
REL_10_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/2bb20e34c11fe1b11f84307a8bb683e865fbbf6d

Modified Files
--------------
src/backend/access/hash/hashfunc.c | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* pgsql: Fix float4/float8 hash functions to produce uniform results for
@ 2021-09-02 21:25 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 7+ messages in thread

From: Tom Lane @ 2021-09-02 21:25 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix float4/float8 hash functions to produce uniform results for NaNs.

The IEEE 754 standard allows a wide variety of bit patterns for NaNs,
of which at least two ("NaN" and "-NaN") are pretty easy to produce
from SQL on most machines.  This is problematic because our btree
comparison functions deem all NaNs to be equal, but our float hash
functions know nothing about NaNs and will happily produce varying
hash codes for them.  That causes unexpected results from queries
that hash a column containing different NaN values.  It could also
produce unexpected lookup failures when using a hash index on a
float column, i.e. "WHERE x = 'NaN'" will not find all the rows
it should.

To fix, special-case NaN in the float hash functions, not too much
unlike the existing special case that forces zero and minus zero
to hash the same.  I arranged for the most vanilla sort of NaN
(that coming from the C99 NAN constant) to still have the same
hash code as before, to reduce the risk to existing hash indexes.

I dithered about whether to back-patch this into stable branches,
but ultimately decided to do so.  It's a clear improvement for
queries that hash internally.  If there is anybody who has -NaN
in a hash index, they'd be well advised to re-index after applying
this patch ... but the misbehavior if they don't will not be much
worse than the misbehavior they had before.

Per bug #17172 from Ma Liangzhu.

Discussion: https://postgr.es/m/17172-7505bea9e04e230f@postgresql.org

Branch
------
REL9_6_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/dea212e245f69fb1cb808e030638b7220311eed9

Modified Files
--------------
src/backend/access/hash/hashfunc.c | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)



^ permalink  raw  reply  [nested|flat] 7+ messages in thread


end of thread, other threads:[~2021-09-02 21:25 UTC | newest]

Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-09-02 21:25 pgsql: Fix float4/float8 hash functions to produce uniform results for Tom Lane <tgl@sss.pgh.pa.us>
2021-09-02 21:25 pgsql: Fix float4/float8 hash functions to produce uniform results for Tom Lane <tgl@sss.pgh.pa.us>
2021-09-02 21:25 pgsql: Fix float4/float8 hash functions to produce uniform results for Tom Lane <tgl@sss.pgh.pa.us>
2021-09-02 21:25 pgsql: Fix float4/float8 hash functions to produce uniform results for Tom Lane <tgl@sss.pgh.pa.us>
2021-09-02 21:25 pgsql: Fix float4/float8 hash functions to produce uniform results for Tom Lane <tgl@sss.pgh.pa.us>
2021-09-02 21:25 pgsql: Fix float4/float8 hash functions to produce uniform results for Tom Lane <tgl@sss.pgh.pa.us>
2021-09-02 21:25 pgsql: Fix float4/float8 hash functions to produce uniform results for Tom Lane <tgl@sss.pgh.pa.us>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox