MIME-Version: 1.0
References: 
 <CACeKOO2hjPmstboJaa=rw8Erd7k5VhXyupU39bAosCPtUe1UBA@mail.gmail.com>
 <CACeKOO1NBExVWG0=wMKc+Kpk-RFXM2i1XTqzGdB9Z_pqg+5otg@mail.gmail.com>
In-Reply-To: 
 <CACeKOO1NBExVWG0=wMKc+Kpk-RFXM2i1XTqzGdB9Z_pqg+5otg@mail.gmail.com>
From: Nadav Shatz <nadav@tailorbrands.com>
Date: Mon, 26 Jan 2026 13:02:40 +0200
Message-ID: 
 <CACeKOO3j9oEXyNiTM+cnjbfr1eaKMEyKf8LM3HTBdVBdrBmVgw@mail.gmail.com>
Subject: Re: Proposal: Recent mutated table tracking in memory
To: pgpool-hackers@lists.postgresql.org
Content-Type: multipart/alternative; boundary="000000000000c17ca306494872b7"
Archived-At: 
 <https://www.postgresql.org/message-id/CACeKOO3j9oEXyNiTM%2Bcnjbfr1eaKMEyKf8LM3HTBdVBdrBmVgw%40mail.gmail.com>
Precedence: bulk

--000000000000c17ca306494872b7
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Doing another ping - sorry for that, it's been 20 days.

Would appreciate any feedback. thank you

On Wed, Jan 14, 2026 at 10:55=E2=80=AFAM Nadav Shatz <nadav@tailorbrands.co=
m> wrote:

> Hi all,
>
> Any comments or concerns? can we merge it if not?
>
> On Tue, Jan 6, 2026 at 1:25=E2=80=AFPM Nadav Shatz <nadav@tailorbrands.co=
m> wrote:
>
>> Hello,
>>
>> As initially proposed under "Proposal: recent access based routing for
>> primary-replica setups" and then broken into separate tasks - i am addin=
g
>> here a patch to implement tracking of latest mutated table, and then usi=
ng
>> the replication lag as a base - deciding where to point queries when que=
ry
>> load balancing and parsing is enabled.
>>
>> More details as in the patch:
>> Feature: add in-memory table tracking to prevent stale reads from replic=
as
>>
>> Implement "memory map" feature that tracks recently-written database
>> tables in shared memory to prevent stale reads during replication lag.
>> When a write (INSERT/UPDATE/DELETE) occurs on a table, that table is
>> marked as "dirty" for a configurable TTL period. Any SELECT on a dirty
>> table within the TTL window is routed to primary instead of replica.
>>
>> Key features:
>> - Shared memory hash table for tracking table mutations with TTL
>> - Query parse cache with LRU eviction for performance
>> - Cold start protection (routes all queries to primary initially)
>> - Automatic TTL calculation: replication_delay =C3=97 configurable facto=
r
>> - Per-table staleness tracking with microsecond precision
>>
>> New configuration parameters:
>> - memory_map_enabled: Enable/disable the feature (default: off)
>> - memory_map_ttl_factor: TTL multiplier for replication delay (default:
>> 5.0)
>> - memory_map_cold_start_duration: Cold start period in ms (default: 2000=
)
>> - memory_map_table_buckets: Hash buckets for table map (default: 1024)
>> - memory_map_table_size: Max tracked tables (default: 2048)
>> - memory_map_query_buckets: Hash buckets for query cache (default: 2048)
>> - memory_map_query_cache_size: Max cached queries (default: 10000)
>>
>> Patch applies properly and tests pass.
>>
>> Open to all feedback - thank you!
>>
>> --
>> Nadav Shatz
>> Tailor Brands | CTO
>>
>
>
> --
> Nadav Shatz
> Tailor Brands | CTO
>


--=20
Nadav Shatz
Tailor Brands | CTO

--000000000000c17ca306494872b7
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Doing another ping - sorry for that, it&#39;s been 20 days=
.<div><br></div><div>Would appreciate any feedback. thank you</div></div><b=
r><div class=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" class=
=3D"gmail_attr">On Wed, Jan 14, 2026 at 10:55=E2=80=AFAM Nadav Shatz &lt;<a=
 href=3D"mailto:nadav@tailorbrands.com">nadav@tailorbrands.com</a>&gt; wrot=
e:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0=
.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"l=
tr">Hi all,<div><br></div><div>Any comments or concerns? can we merge it if=
 not?</div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"g=
mail_attr">On Tue, Jan 6, 2026 at 1:25=E2=80=AFPM Nadav Shatz &lt;<a href=
=3D"mailto:nadav@tailorbrands.com" target=3D"_blank">nadav@tailorbrands.com=
</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:=
0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">=
<div dir=3D"ltr"><div>Hello,</div><div><br></div><div>As initially proposed=
 under &quot;Proposal: recent access based routing for primary-replica setu=
ps&quot; and then broken into separate=C2=A0tasks - i am adding here a patc=
h to implement tracking of latest mutated table, and then using the replica=
tion lag as a base - deciding where to point queries when query load balanc=
ing and parsing is enabled.</div><div><br></div><div>More details as in the=
 patch:</div><div>Feature: add in-memory table tracking to prevent stale re=
ads from replicas<br><br>Implement &quot;memory map&quot; feature that trac=
ks recently-written database<br>tables in shared memory to prevent stale re=
ads during replication lag.<br>When a write (INSERT/UPDATE/DELETE) occurs o=
n a table, that table is<br>marked as &quot;dirty&quot; for a configurable =
TTL period. Any SELECT on a dirty<br>table within the TTL window is routed =
to primary instead of replica.<br><br>Key features:<br>- Shared memory hash=
 table for tracking table mutations with TTL<br>- Query parse cache with LR=
U eviction for performance<br>- Cold start protection (routes all queries t=
o primary initially)<br>- Automatic TTL calculation: replication_delay =C3=
=97 configurable factor<br>- Per-table staleness tracking with microsecond =
precision<br><br>New configuration parameters:<br>- memory_map_enabled: Ena=
ble/disable the feature (default: off)<br>- memory_map_ttl_factor: TTL mult=
iplier for replication delay (default: 5.0)<br>- memory_map_cold_start_dura=
tion: Cold start period in ms (default: 2000)<br>- memory_map_table_buckets=
: Hash buckets for table map (default: 1024)<br>- memory_map_table_size: Ma=
x tracked tables (default: 2048)<br>- memory_map_query_buckets: Hash bucket=
s for query cache (default: 2048)<br>- memory_map_query_cache_size: Max cac=
hed queries (default: 10000)</div><div><br></div><div>Patch applies properl=
y and tests pass.</div><div><br></div><div>Open to all feedback - thank you=
!</div><div><br></div><span class=3D"gmail_signature_prefix">-- </span><br>=
<div dir=3D"rtl" class=3D"gmail_signature"><div dir=3D"ltr"><div><div dir=
=3D"ltr"><div dir=3D"rtl"><div dir=3D"ltr"><font color=3D"#000000">Nadav Sh=
atz</font></div><div dir=3D"ltr"><font color=3D"#000000">Tailor Brands=C2=
=A0| CTO</font></div></div></div></div></div></div></div>
</blockquote></div><div><br clear=3D"all"></div><div><br></div><span class=
=3D"gmail_signature_prefix">-- </span><br><div dir=3D"rtl" class=3D"gmail_s=
ignature"><div dir=3D"ltr"><div><div dir=3D"ltr"><div dir=3D"rtl"><div dir=
=3D"ltr"><font color=3D"#000000">Nadav Shatz</font></div><div dir=3D"ltr"><=
font color=3D"#000000">Tailor Brands=C2=A0| CTO</font></div></div></div></d=
iv></div></div>
</blockquote></div><div><br clear=3D"all"></div><div><br></div><span class=
=3D"gmail_signature_prefix">-- </span><br><div dir=3D"rtl" class=3D"gmail_s=
ignature"><div dir=3D"ltr"><div><div dir=3D"ltr"><div dir=3D"rtl"><div dir=
=3D"ltr"><font color=3D"#000000">Nadav Shatz</font></div><div dir=3D"ltr"><=
font color=3D"#000000">Tailor Brands=C2=A0| CTO</font></div></div></div></d=
iv></div></div>

--000000000000c17ca306494872b7--