From: Fabien COELHO <coelho@cri.ensmp.fr>
To: Martin Kalcher <martin.kalcher@aboutsource.net>
Cc: Dean Rasheed <dean.a.rasheed@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Andrew Dunstan <andrew@dunslane.net>
Cc: John Naylor <john.naylor@enterprisedb.com>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: [PATCH] Introduce array_shuffle() and array_sample()
Date: Sun, 24 Jul 2022 21:42:48 +0200 (CEST)
Message-ID: <alpine.DEB.2.22.394.2207242132460.1359119@pseudo> (raw)
In-Reply-To: <f5af7698-d1eb-bdf4-9969-82b18ab4f8cf@aboutsource.net>
References: <9d160a44-7675-51e8-60cf-6d64b76db831@aboutsource.net>
<809171.1658176490@sss.pgh.pa.us>
<817328.1658178181@sss.pgh.pa.us>
<0b5dea2b-37fa-6c81-ea9d-0341ab6f87d6@aboutsource.net>
<091a9f69-1e5d-6558-41a7-2607a5fc9ff7@aboutsource.net>
<CA+TgmoYqC42KS7mypjMKxvGY2oPYz=_kvf1zcDNdawXXiVXqQA@mail.gmail.com>
<1064800.1658240423@sss.pgh.pa.us>
<7e7cf274-09cb-00cf-1715-979f011e40b0@aboutsource.net>
<CAEZATCUPcdxeRLmiFvuzZ=kKVpx8kOKTYDPY4oVv3hQBJYfsCQ@mail.gmail.com>
<f6267e99-4448-b47c-9fe8-ec23690b931f@aboutsource.net>
<CAEZATCVOJHxWOG7DvEjkL_Y-h6f-HK-VkFpsdP76pa6y7xsW7Q@mail.gmail.com>
<459f6008-90bc-bc52-6f11-e68570e03ac0@aboutsource.net>
<CAEZATCXCFyr_cpkMygqSiiiHXgZWaXSumwL9=Xz5n7wwzBvaEw@mail.gmail.com>
<583b6409-df52-86d1-8c0a-ce8e74863ad7@aboutsource.net>
<036ba529-1f25-d7b5-c5ad-e6c852d928df@aboutsource.net>
<alpine.DEB.2.22.394.2207240929220.1359119@pseudo>
<f5af7698-d1eb-bdf4-9969-82b18ab4f8cf@aboutsource.net>
Hello,
> Thank you for your feedback. I attached a patch, that addresses most of your
> points.
I'll look into it. It would help if the patch could include a version
number at the end.
>> Should the exchange be skipped when i == k?
>
> The additional branch is actually slower (on my machine, tested with an 10M
> element int array) for 1D-arrays, which i assume is the most common case.
> Even with a 2D-array with a subarray size of 1M there is no difference in
> execution time. i == k should be relatively rare for large arrays, given a
> good prng.
Ok, slower is bad.
>> The random and array shuffling functions share a common state. I'm
>> wondering whether it should ot should not be so. It seems ok, but then ISTM
>> that the documentation suggests implicitely that setseed applies to
>> random() only, which is not the case anymore after the patch.
>
> I really like the idea of a prng state owned by the user, that is used by all
> user facing random functions, but see that their might be concerns about this
> change. I would update the setseed() documentation, if this proposal is
> accepted. Do you think my patch should already include the documentation
> update?
Duno. I'm still wondering what it should do. I'm pretty sure that the
documentation should be clear about a shared seed, if any. I do not think
that departing from the standard is a good thing, either.
>> If more samples are required than the number of elements, it does not error
>> out. I'm wondering whether it should.
>
> The second argument to array_sample() is treated like a LIMIT, rather than
> the actual number of elements. I updated the documentation. My use-case is:
> take max random items from an array of unknown size.
Hmmm. ISTM that the use case of wanting "this many" stuff would make more
sense because it is strictier so less likely to result in unforseen
consequences. On principle I do not like not knowing the output size.
If someone wants a limit, they can easily "LEAST(#1 dim size, other
limit)" to get it, it is easy enough with a strict function.
>> Also, the sampling should not return its result in order when the number of
>> elements required is the full array, ISTM that it should be shuffled there
>> as well.
>
> You are the second person asking for this. It's done. I am thinking about
> ditching array_sample() and replace it with a second signature for
> array_shuffle():
>
> array_shuffle(array anyarray) -> anyarray
> array_shuffle(array anyarray, limit int) -> anyarray
>
> What do you think?
I think that shuffle means shuffle, not partial shuffle, so a different
name seems better to me.
--
Fabien.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-general@postgresql.org
Cc: coelho@cri.ensmp.fr, martin.kalcher@aboutsource.net, dean.a.rasheed@gmail.com, tgl@sss.pgh.pa.us, robertmhaas@gmail.com, andrew@dunslane.net, john.naylor@enterprisedb.com, thomas.munro@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: [PATCH] Introduce array_shuffle() and array_sample()
In-Reply-To: <alpine.DEB.2.22.394.2207242132460.1359119@pseudo>
References: <9d160a44-7675-51e8-60cf-6d64b76db831@aboutsource.net> <809171.1658176490@sss.pgh.pa.us> <817328.1658178181@sss.pgh.pa.us> <0b5dea2b-37fa-6c81-ea9d-0341ab6f87d6@aboutsource.net> <091a9f69-1e5d-6558-41a7-2607a5fc9ff7@aboutsource.net> <CA+TgmoYqC42KS7mypjMKxvGY2oPYz=_kvf1zcDNdawXXiVXqQA@mail.gmail.com> <1064800.1658240423@sss.pgh.pa.us> <7e7cf274-09cb-00cf-1715-979f011e40b0@aboutsource.net> <CAEZATCUPcdxeRLmiFvuzZ=kKVpx8kOKTYDPY4oVv3hQBJYfsCQ@mail.gmail.com> <f6267e99-4448-b47c-9fe8-ec23690b931f@aboutsource.net> <CAEZATCVOJHxWOG7DvEjkL_Y-h6f-HK-VkFpsdP76pa6y7xsW7Q@mail.gmail.com> <459f6008-90bc-bc52-6f11-e68570e03ac0@aboutsource.net> <CAEZATCXCFyr_cpkMygqSiiiHXgZWaXSumwL9=Xz5n7wwzBvaEw@mail.gmail.com> <583b6409-df52-86d1-8c0a-ce8e74863ad7@aboutsource.net> <036ba529-1f25-d7b5-c5ad-e6c852d928df@aboutsource.net> <alpine.DEB.2.22.394.2207240929220.1359119@pseudo> <f5af7698-d1eb-bdf4-9969-82b18ab4f8cf@aboutsource.net> <alpine.DEB.2.22.394.2207242132460.1359119@pseudo>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox