MIME-Version: 1.0
References: <CAM+6J97Jd5WnxdyFu1-s+wvJrWPL_YvP3yeuQVwy3HJvK=KUTQ@mail.gmail.com>
 <202411131029.qchduffwgzhm@alvherre.pgsql> <CAFCRh-9Le3fkJMWGuHdFPP-kfHdEAAchFZf0msEq3hNijixbFw@mail.gmail.com>
 <7d32e088-34a7-421a-9398-80958acb3f64@technowledgy.de>
In-Reply-To: <7d32e088-34a7-421a-9398-80958acb3f64@technowledgy.de>
From: Eric Hanson <eric@aquameta.com>
Date: Fri, 22 Nov 2024 06:24:10 -0600
Message-ID: <CACA6kxh5EWZPMR5XCEXh3C-t-bBfdhbge1z+nGFpCOUgbZK+EA@mail.gmail.com>
Subject: Re: Fwd: A million users
To: walther@technowledgy.de
Cc: Dominique Devienne <ddevienne@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, 
	Vijaykumar Jain <vijaykumarjain.github@gmail.com>, 
	pgsql-general <pgsql-general@postgresql.org>, "kaare@jasonic.dk" <kaare@jasonic.dk>
Content-Type: multipart/alternative; boundary="00000000000075306d06277f764e"
Archived-At: <https://www.postgresql.org/message-id/CACA6kxh5EWZPMR5XCEXh3C-t-bBfdhbge1z%2BnGFpCOUgbZK%2BEA%40mail.gmail.com>
Precedence: bulk

--00000000000075306d06277f764e
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Nov 13, 2024 at 12:02=E2=80=AFPM <walther@technowledgy.de> wrote:

> I don't have any benchmarks, but the following data point:
>
> We use PostgREST [1] which connects to the database with one
> "authenticator" role. For each request it handles, it does a SET ROLE to
> a role defined in the Authorization Header (JWT).
>
> Some numbers we are running with currently:
> - 1 authenticator role
> - ~ 127.000 user roles granted to "authenticator"
> - ~ 14.000 "scope" roles granted to the user roles (tenants, groups, ..)
> - ~ 15 "context" roles granted to user roles ("admin", "user", ...)
> - ~ 50 "access" roles granted to context roles ("view_x", "do_y", ...)
>
> Only the access roles have any direct privileges granted.
>
> We currently have ~ 700 RLS policies defined. Those are created TO the
> context roles. The policies check the current role's scope roles to
> select "allowed" rows.
>
> In total, we have ~370.000 roles granted to each other (pg_auth_members).
>
> Except for one thing, we have never had any real problems with this. We
> didn't observe anything getting massively worse with many roles, even
> though we use them extensively. RLS policies need to be carefully
> written to get any performance, though.
>
> The one problem we found is:
>
> The first time the authenticator role does a SET ROLE in a session it's
> **terribly** slow. With fewer users back then it took 6-7 minutes to do
> it. Any SET ROLE afterwards in the same session would be fast. Even more
> annoying - killing the session with SET ROLE running would not work
> properly and leave zombie processes. Giving the authenticator role the
> SUPERUSER privilege avoids the problem and makes it instant. However..
> that's not very desirable.
>
> There were some improvements, IIRC in the 17 cycle (?), in that area,
> but I had not have the time to test it with that. We are still on v15
> and the last time I tested this was ~ two years ago. I still wasn't able
> to put together a simple reproducer either.
>
> You should *probably* be better off with your different LOGIN roles, I
> assume the role cache builds up much quicker in that case.
>

I'm really interested in how this works.  Role-per-user or even the ability
to have many roles (370k??) seems like a dream come true. But I always was
wary of it because:

a) A connection-per-role hits the ceiling pretty quickly because
connections can't be pooled and shared between users and take up a lot of
memory etc.

b) One could try to get around this with an authenticator role as you
describe, but isn't it then possible to do a RESET ROLE and then another
SET ROLE to get access to another user?  This of course would have to be
through SQL injection or some such, but it seems like that defeats at least
some of the purpose of RLS.

Did you find some way to prevent RESET ROLE?  I once advocated for a NO
RESET option on SET ROLE [1] so that RESET ROLE would be impossible for the
rest of the session.  Still think it would be helpful.

Thanks,
Eric

[1]
https://www.postgresql.org/message-id/flat/CACA6kxgdzt-oForijaxfXHHhnZ1WBoV=
GMXVwFrJqUu-Hg3C-jA%40mail.gmail.com

--00000000000075306d06277f764e
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Wed, Nov 13, 2024 at 12:02=E2=80=AFPM =
&lt;<a href=3D"mailto:walther@technowledgy.de">walther@technowledgy.de</a>&=
gt; wrote:</div><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote=
" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);=
padding-left:1ex">I don&#39;t have any benchmarks, but the following data p=
oint:<br>
<br>
We use PostgREST [1] which connects to the database with one <br>
&quot;authenticator&quot; role. For each request it handles, it does a SET =
ROLE to <br>
a role defined in the Authorization Header (JWT).<br>
<br>
Some numbers we are running with currently:<br>
- 1 authenticator role<br>
- ~ 127.000 user roles granted to &quot;authenticator&quot;<br>
- ~ 14.000 &quot;scope&quot; roles granted to the user roles (tenants, grou=
ps, ..)<br>
- ~ 15 &quot;context&quot; roles granted to user roles (&quot;admin&quot;, =
&quot;user&quot;, ...)<br>
- ~ 50 &quot;access&quot; roles granted to context roles (&quot;view_x&quot=
;, &quot;do_y&quot;, ...)<br>
<br>
Only the access roles have any direct privileges granted.<br>
<br>
We currently have ~ 700 RLS policies defined. Those are created TO the <br>
context roles. The policies check the current role&#39;s scope roles to <br=
>
select &quot;allowed&quot; rows.<br>
<br>
In total, we have ~370.000 roles granted to each other (pg_auth_members).<b=
r>
<br>
Except for one thing, we have never had any real problems with this. We <br=
>
didn&#39;t observe anything getting massively worse with many roles, even <=
br>
though we use them extensively. RLS policies need to be carefully <br>
written to get any performance, though.<br>
<br>
The one problem we found is:<br>
<br>
The first time the authenticator role does a SET ROLE in a session it&#39;s=
 <br>
**terribly** slow. With fewer users back then it took 6-7 minutes to do <br=
>
it. Any SET ROLE afterwards in the same session would be fast. Even more <b=
r>
annoying - killing the session with SET ROLE running would not work <br>
properly and leave zombie processes. Giving the authenticator role the <br>
SUPERUSER privilege avoids the problem and makes it instant. However.. <br>
that&#39;s not very desirable.<br>
<br>
There were some improvements, IIRC in the 17 cycle (?), in that area, <br>
but I had not have the time to test it with that. We are still on v15 <br>
and the last time I tested this was ~ two years ago. I still wasn&#39;t abl=
e <br>
to put together a simple reproducer either.<br>
<br>
You should *probably* be better off with your different LOGIN roles, I <br>
assume the role cache builds up much quicker in that case.<br></blockquote>=
<div><br></div><div>I&#39;m really interested in how this works.=C2=A0 Role=
-per-user or even the ability to have many roles (370k??)=C2=A0seems like a=
 dream come true.=C2=A0But I always was wary of it because:</div><div><br><=
/div><div>a) A connection-per-role hits the ceiling pretty quickly because =
connections can&#39;t be pooled and shared between users and take up a lot =
of memory etc.</div><div><br></div><div>b) One could try to get around this=
 with an authenticator role as you describe, but isn&#39;t it then possible=
 to do a RESET ROLE and then another SET ROLE to get access to another user=
?=C2=A0 This of course would have to be through SQL injection or some such,=
 but it seems like that defeats at least some of the purpose of RLS.</div><=
div><br></div><div>Did you find some way to prevent RESET ROLE?=C2=A0 I onc=
e advocated for a NO RESET option on SET ROLE [1] so that RESET ROLE would =
be impossible for the rest of the session.=C2=A0 Still think it would be he=
lpful.</div><div><br></div><div>Thanks,</div><div>Eric</div><div><br></div>=
<div>[1]=C2=A0<a href=3D"https://www.postgresql.org/message-id/flat/CACA6kx=
gdzt-oForijaxfXHHhnZ1WBoVGMXVwFrJqUu-Hg3C-jA%40mail.gmail.com" target=3D"_b=
lank">https://www.postgresql.org/message-id/flat/CACA6kxgdzt-oForijaxfXHHhn=
Z1WBoVGMXVwFrJqUu-Hg3C-jA%40mail.gmail.com</a></div><div>=C2=A0</div></div>=
</div>

--00000000000075306d06277f764e--