Feedback-ID: ic6394509:Fastmail
User-Agent: Cyrus-JMAP/3.11.0-alpha0-566-g3812ddbbc-fm-20240627.001-g3812ddbb
MIME-Version: 1.0
Message-Id: <b2079087-86be-43c3-a298-6bc0b49e2d24@app.fastmail.com>
In-Reply-To: <018dd6e8-8f21-4b3b-8d80-2d128f6d00b4@app.fastmail.com>
References: <44d2ffca-d560-4919-b85a-4d07060946aa@app.fastmail.com>
 <87zfr1e1uj.fsf@wibble.ilmari.org>
 <9ff90ff9-5ac1-4529-b2f4-f93b37734b9c@app.fastmail.com>
 <81348268-b9fd-4308-bf52-056070aa7c2a@app.fastmail.com>
 <f25ddfc8-4f02-4e8f-8e74-0532f48d0817@app.fastmail.com>
 <CAEZATCV2qPTGo2Fd8xDs06Q7iU5aorgSa9+Fw9zkuQv1y15rcw@mail.gmail.com>
 <8e909218-1965-4515-99e3-4bb5b625e004@app.fastmail.com>
 <CAEZATCWEou=NjpvS-r1WxymbAYihsdKQoQBj1HfBeocoNcv5Ww@mail.gmail.com>
 <bbc62c2d-1173-4454-9e17-be99437c0e7e@app.fastmail.com>
 <0459ded5-6daf-4bf0-9ab0-e1f11ab15ec6@app.fastmail.com>
 <CAEZATCW+N9qaE=5_1xcB0isMrirojN=Bd0yY4Pw67iji4swYiQ@mail.gmail.com>
 <4e54bf4f-59fe-4c9a-b446-a2b64cbfd584@app.fastmail.com>
 <fb6e4aa6-54e1-4be4-a7f6-3edc99aa7e6e@app.fastmail.com>
 <27d42b25-f328-4231-9db0-3811dc77c8cf@app.fastmail.com>
 <33c51978-8ee8-438c-9804-e46c0ca8f3f9@app.fastmail.com>
 <CAEZATCUA+GBYvzbqSr7LPDDfmr24jQE1R0iyYfjFk82xeuBCRw@mail.gmail.com>
 <018dd6e8-8f21-4b3b-8d80-2d128f6d00b4@app.fastmail.com>
Date: Wed, 03 Jul 2024 21:05:51 +0200
From: "Joel Jacobson" <joel@compiler.org>
To: "Dean Rasheed" <dean.a.rasheed@gmail.com>
Cc: =?UTF-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= <ilmari@ilmari.org>,
 pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Optimize numeric multiplication for one and two base-NBASE digit
 multiplicands.
Content-Type: text/plain
Archived-At: <https://www.postgresql.org/message-id/b2079087-86be-43c3-a298-6bc0b49e2d24%40app.fastmail.com>
Precedence: bulk

On Wed, Jul 3, 2024, at 15:48, Joel Jacobson wrote:
> On Wed, Jul 3, 2024, at 13:17, Dean Rasheed wrote:
>> On Tue, 2 Jul 2024 at 21:10, Joel Jacobson <joel@compiler.org> wrote:
>>>
>>> I found the bug in the case 3 code,
>>> and it turns out the same type of bug also exists in the case 2 code:
>>>
>>>                         case 2:
>>>                                 newdig = (int) var1digits[1] * var2digits[res_ndigits - 4];
>>>
>>> The problem here is that res_ndigits could become less than 4,
>>
>> Yes. It can't be less than 3 though (per an earlier test), so the case
>> 2 code was correct.
>
> Hmm, I don't see how the case 2 code can be correct?
> If, like you say, res_ndigits can't be less than 3, that means it can 
> be 3, right?
> And if res_ndigits=3 then `var2digits[res_ndigits - 4]` would try to 
> access `var2digits[-1]`.

Here is an example on how to trigger the bug:

```
			case 2:
				if (res_ndigits - 4 < 0)
				{
					printf("var1=%s\n",get_str_from_var(var1));
					printf("var2=%s\n",get_str_from_var(var2));
					printf("rscale=%d\n", rscale);
					printf("res_ndigits - 4 < 0 => var2digits[%d]=%d\n", res_ndigits - 4, var2digits[res_ndigits - 4]);
				}
```

Running through my tests, I hit lots of cases, including:

var1=0.10968501
var2=0.903728177113
rscale=0
res_ndigits - 4 < 0 => var2digits[-1]=-31105

All of the spotted cases had rscale=0.

If we know that mul_var() will never be called with rscale=0 when dealing with decimal inputs, perhaps we should enforce this with an Assert(), to prevent the otherwise possible out-of-bounds access (negative indexing) and provide early detection?

Regards,
Joel