Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1u4qIS-0084VS-EB for pgsql-hackers@arkaria.postgresql.org; Wed, 16 Apr 2025 00:06:33 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1u4qIQ-001tse-Lz for pgsql-hackers@arkaria.postgresql.org; Wed, 16 Apr 2025 00:06:31 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1u4qIQ-001tsI-5q for pgsql-hackers@lists.postgresql.org; Wed, 16 Apr 2025 00:06:31 +0000 Received: from mail-ed1-x535.google.com ([2a00:1450:4864:20::535]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1u4qIM-000IZ5-2C for pgsql-hackers@lists.postgresql.org; Wed, 16 Apr 2025 00:06:30 +0000 Received: by mail-ed1-x535.google.com with SMTP id 4fb4d7f45d1cf-5e5c9662131so9509075a12.3 for ; Tue, 15 Apr 2025 17:06:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1744761985; x=1745366785; darn=lists.postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=v8jRXyEd17pNkh0HdhO4HDu8S5hkygNg3FnyG5lnE0Q=; b=CxTOcBGeEFJ2dovmavGujCWcoX+RYkSeBsyXMrx16zNl4iU6cilQX66sFly/Awrbjp x7k+cyZI9K1OKvAcX7VPP08l3/Cy2eUgFtNME10AjPee9u7RQn0ynb87EkgXmzLB4vci 2m1W2WQybscIdY0BZn0Ge5VxzHwDZ6vOVXdOk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744761985; x=1745366785; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=v8jRXyEd17pNkh0HdhO4HDu8S5hkygNg3FnyG5lnE0Q=; b=PBMtkCxWQK8n2OsUJcv17DdbwZfuGuHNoPRFmR4aizx3B85LcNefPh0p6u+M9JIesR rvEt+Su1dqBYpFurQql/cjtmqUtSzBLllAYRmnKpLIOe9coIAQ37J7JQLaIBHc91iz4x v0NUa8Rp/5Wqo4VczQrImcTYKe1OlgmznUvjWt8xzB+ruuo/7xGKzNnNXAsgFa2t4fvG KpwURhh3HxhKaAe60Je5Jt0cWNRAoY/9VUfeGM/iBiN12RxOOttQgU6d1FdqNMlWh5nk AlCr4NpYKV+7AFVQulQxze9Ihs3r5ZZgkiLhGVe0sXWqn+q0/59C9eXAuILUzT3OwTe5 eDuQ== X-Forwarded-Encrypted: i=1; AJvYcCU5G+nrcslZMcVhzs1zV/CFgRvJz4UX+THySQqDy+9BsjttVDsgf6KtMzXu3cc+5H97MjRLEmQZZxzyANYz@lists.postgresql.org X-Gm-Message-State: AOJu0YzmaQMZv9VLnl7Ci4iRrLiozlq3HZAlPAqu70L4mrL5q1s56vgF L8yNDc5wziygO/+QWKufkVypZO4gaO4fuSOnh03tfJmpogTNNX9BAZ4ozsAEAQmZog8IGj5dElD rJUprRVfu/6V7/rjsoaBGT5nLO6GGcIY6/XBavl+c+PTWZxdfQ4JVwqy/vJg5YH5J34TCx6c3dP UdEwOg7U5OZx79vG1C2iRDIeSyK++CnypERvN442D0VaDqL6s= X-Gm-Gg: ASbGncsKmy7rxHQlpPv60S1+OUmyPnqvatUHruJCtb70qVteILz0gxbCWB54VMJOkwn bepG2ckStDI7Q4oZoVejrdSRLYxQ9mFaWU+ye0BoEFvSEMaT1WnAtPRzQnAoRSdTcn2yLJsGfCh rOjMhGmQlzl1C7OMvxNxMnsfcsp81KUMXn7kPs+kJ2ToWtGJKsKLn+gkA= X-Google-Smtp-Source: AGHT+IEZqr+cUo/jmP7CiTNPUQm8SWm93xGczJ0j/4nElqti7zeE4a10h5cROqfOvwYPezeugJzB5nTTJEgtwHpzu/g= X-Received: by 2002:a05:6402:5407:b0:5f1:6a55:3941 with SMTP id 4fb4d7f45d1cf-5f49a1da7cfmr772661a12.14.1744761984756; Tue, 15 Apr 2025 17:06:24 -0700 (PDT) MIME-Version: 1.0 References: <2531459.1743871597@sss.pgh.pa.us> In-Reply-To: From: Ashwin Agrawal Date: Tue, 15 Apr 2025 17:06:13 -0700 X-Gm-Features: ATxdqUEWo2NyGq56-e0-4Qa7LYoNRVxyKcCVAZeeQMbHKiuKHqzHFMOWGOZp1Bk Message-ID: Subject: Re: A modest proposal: make parser/rewriter/planner inputs read-only To: Andres Freund Cc: Tom Lane , pgsql-hackers@lists.postgresql.org Content-Type: multipart/alternative; boundary="00000000000051cc370632da0ee9" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --00000000000051cc370632da0ee9 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Apr 5, 2025 at 7:31=E2=80=AFPM Andres Freund w= rote: > Hi, > > On 2025-04-05 12:46:37 -0400, Tom Lane wrote: > > 1. Invent a way to make a particular memory context read-only > > after putting some data into it. > > > > 2. In debug builds, after we've built a tree that should be considered > > read-only, copy it into such a context and make it read-only. Or > > perhaps build it there in the first place. > > > 3. Fix the resulting crashes. > > > > 4. Profit! (In particular, nuke a lot of no-longer-needed copyObject > > calls.) > > > > My first thought about implementing #1 was to seek Valgrind's help, > > but so far as I can find out there's no VALGRIND_MAKE_MEM_READ_ONLY. > > Step #3 would be pretty tedious anyway if it required running under > > Valgrind. However, all modern hardware has the ability to mark > > memory read-only at the page level, and most platforms expose that > > in some way or other. So it doesn't seem unreasonable to invent > > a memory context option (or whole new context type, if that seems > > easier) that is careful to align its allocation blocks on page > > boundaries and then can set or clear the hardware R/O flag on > > demand. It'd be enough if the R/O enforcement worked on popular > > development platforms, we don't have to make it work absolutely > > everywhere. > > FWIW, while hacking on patch to making hint bit writes not happening whil= e > IO > is going on (so we don't need to copy the page anymore and don't cause > filesystem level issues with DIO), I hacked up protection for shared > buffers > using mprotect() - it worked way better than I thought it would. The > overhead > ended up surprisingly low: > > base: > real 1m4.613s > user 4m31.409s > sys 3m20.445s > > ENFORCE_BUFFER_PROT > > real 1m11.912s > user 4m27.332s > sys 3m28.063s > > > See https://postgr.es/m/043c8b50-d183-46e5-b054-145cc0f6f908%40iki.fi > > > I'm mostly sharing that to say that > a) yes, mprotect() is viable and works surprisingly well > b) it might be worth inventing some common platform abstraction for > mprotect > > That prototype patch already worked on most platforms, windows should be > entirely doable. Also, I would like to provide reference to this old thread [1] in favor of mprotect(). In that thread, and in Greenplum using mprotect helps detect Shared buffer access rule violations. [1] https://www.postgresql.org/message-id/CANXE4Tfxmjv8Z-kcwWUCYxx6zCFS%3DGeBPm= -ZjjLucvrGewjUtg%40mail.gmail.com --=20 This electronic communication and the information and any files transmitted= =20 with it, or attached to it, are confidential and are intended solely for=20 the use of the individual or entity to whom it is addressed and may contain= =20 information that is confidential, legally privileged, protected by privacy= =20 laws, or otherwise restricted from disclosure to anyone else. If you are=20 not the intended recipient or the person responsible for delivering the=20 e-mail to the intended recipient, you are hereby notified that any use,=20 copying, distributing, dissemination, forwarding, printing, or copying of= =20 this e-mail is strictly prohibited. If you received this e-mail in error,= =20 please return the e-mail to the sender, delete it from your computer, and= =20 destroy any printed copy of it. --00000000000051cc370632da0ee9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Sat, Apr 5, 2025 at 7:31=E2=80=AFPM An= dres Freund <andres@anarazel.de> wrote:
Hi,

On 2025-04-05 12:46:37 -0400, Tom Lane wrote:
> 1. Invent a way to make a particular memory context read-only
> after putting some data into it.
>
> 2. In debug builds, after we've built a tree that should be consid= ered
> read-only, copy it into such a context and make it read-only.=C2=A0 Or=
> perhaps build it there in the first place.

> 3. Fix the resulting crashes.
>
> 4. Profit!=C2=A0 (In particular, nuke a lot of no-longer-needed copyOb= ject
> calls.)
>
> My first thought about implementing #1 was to seek Valgrind's help= ,
> but so far as I can find out there's no VALGRIND_MAKE_MEM_READ_ONL= Y.
> Step #3 would be pretty tedious anyway if it required running under > Valgrind.=C2=A0 However, all modern hardware has the ability to mark > memory read-only at the page level, and most platforms expose that
> in some way or other.=C2=A0 So it doesn't seem unreasonable to inv= ent
> a memory context option (or whole new context type, if that seems
> easier) that is careful to align its allocation blocks on page
> boundaries and then can set or clear the hardware R/O flag on
> demand.=C2=A0 It'd be enough if the R/O enforcement worked on popu= lar
> development platforms, we don't have to make it work absolutely > everywhere.

FWIW, while hacking on patch to making hint bit writes not happening while = IO
is going on (so we don't need to copy the page anymore and don't ca= use
filesystem level issues with DIO), I hacked up protection for shared buffer= s
using mprotect() - it worked way better than I thought it would. The overhe= ad
ended up surprisingly low:

base:
real=C2=A0 =C2=A0 1m4.613s
user=C2=A0 =C2=A0 4m31.409s
sys=C2=A0 =C2=A0 =C2=A03m20.445s

ENFORCE_BUFFER_PROT

real=C2=A0 =C2=A0 1m11.912s
user=C2=A0 =C2=A0 4m27.332s
sys=C2=A0 =C2=A0 =C2=A03m28.063s


See https://postgr.es/m/043c8b50-d1= 83-46e5-b054-145cc0f6f908%40iki.fi


I'm mostly sharing that to say that
a) yes, mprotect() is viable and works surprisingly well
b) it might be worth inventing some common platform abstraction for mprotec= t

That prototype patch already worked on most platforms, windows should be entirely doable.

Also, I would like to prov= ide reference to this old thread [1] in favor of mprotect().
In t= hat thread, and in Greenplum using mprotect helps detect Shared buffer acce= ss rule violations.


This ele= ctronic communication and the information and any files transmitted with it= , or attached to it, are confidential and are intended solely for the use o= f the individual or entity to whom it is addressed and may contain informat= ion that is confidential, legally privileged, protected by privacy laws, or= otherwise restricted from disclosure to anyone else. If you are not the in= tended recipient or the person responsible for delivering the e-mail to the= intended recipient, you are hereby notified that any use, copying, distrib= uting, dissemination, forwarding, printing, or copying of this e-mail is st= rictly prohibited. If you received this e-mail in error, please return the = e-mail to the sender, delete it from your computer, and destroy any printed= copy of it. --00000000000051cc370632da0ee9--