pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Use streaming read I/O in VACUUM's first phase
4+ messages / 1 participants
[nested] [flat]
* pgsql: Use streaming read I/O in VACUUM's first phase
@ 2025-02-14 17:59 Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 0 replies; 4+ messages in thread
From: Melanie Plageman @ 2025-02-14 17:59 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Use streaming read I/O in VACUUM's first phase
Make vacuum's first phase, which prunes and freezes tuples and records
dead TIDs, use the read stream API by by converting
heap_vac_scan_next_block() to a read stream callback.
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CAAKRu_aLwANZpxHc0tC-6OT0OQT4TftDGkKAO5yigMUOv_Tcsw%40mail.gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/9256822608f3b0636cfd88c3436cdceda7003cf3
Modified Files
--------------
src/backend/access/heap/vacuumlazy.c | 152 ++++++++++++++++++++++-------------
1 file changed, 94 insertions(+), 58 deletions(-)
^ permalink raw reply [nested|flat] 4+ messages in thread
* pgsql: Use streaming read I/O in VACUUM's third phase
@ 2025-02-14 17:59 Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 4+ messages in thread
From: Melanie Plageman @ 2025-02-14 17:59 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Use streaming read I/O in VACUUM's third phase
Make vacuum's third phase (its second pass over the heap), which reaps
dead items collected in the first phase and marks them as reusable, use
the read stream API. This commit adds a new read stream callback,
vacuum_reap_lp_read_stream_next(), that looks ahead in the TidStore and
returns the next block number to read for vacuum.
Author: Melanie Plageman <melanieplageman@gmail.com>
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGKN3oy0bN_3yv8hd78a4%2BM1tJC9z7mD8%2Bf%2ByA%2BGeoFUwQ%40mail.gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/c3e775e608f2a6d0bcfba147bf08a506827cc567
Modified Files
--------------
src/backend/access/heap/vacuumlazy.c | 55 ++++++++++++++++++++++++++++++++----
1 file changed, 49 insertions(+), 6 deletions(-)
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: pgsql: Use streaming read I/O in VACUUM's third phase
@ 2025-02-14 18:31 Melanie Plageman <melanieplageman@gmail.com>
parent: Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 4+ messages in thread
From: Melanie Plageman @ 2025-02-14 18:31 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
On Fri, Feb 14, 2025 at 12:59 PM Melanie Plageman
<melanieplageman@gmail.com> wrote:
>
> Use streaming read I/O in VACUUM's third phase
>
> Make vacuum's third phase (its second pass over the heap), which reaps
> dead items collected in the first phase and marks them as reusable, use
> the read stream API. This commit adds a new read stream callback,
> vacuum_reap_lp_read_stream_next(), that looks ahead in the TidStore and
> returns the next block number to read for vacuum.
>
> Author: Melanie Plageman <melanieplageman@gmail.com>
> Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
> Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
> Discussion: https://postgr.es/m/CA%2BhUKGKN3oy0bN_3yv8hd78a4%2BM1tJC9z7mD8%2Bf%2ByA%2BGeoFUwQ%40mail.gmail.com
>
> Branch
> ------
> master
>
> Details
> -------
> https://git.postgresql.org/pg/commitdiff/c3e775e608f2a6d0bcfba147bf08a506827cc567
>
> Modified Files
> --------------
> src/backend/access/heap/vacuumlazy.c | 55 ++++++++++++++++++++++++++++++++----
> 1 file changed, 49 insertions(+), 6 deletions(-)
I'm looking into the valgrind failures [1].
==1526248== VALGRINDERROR-END
{
<insert_a_suppression_name_here>
Memcheck:Addr1
fun:lazy_scan_heap
fun:heap_vacuum_rel
fun:table_relation_vacuum
fun:vacuum_rel
fun:vacuum_rel
fun:vacuum
fun:ExecVacuum
fun:standard_ProcessUtility
fun:ProcessUtility
fun:PortalRunUtility
fun:PortalRunMulti
fun:PortalRun
fun:exec_simple_query
}
**1526248** Valgrind detected 492 error(s) during execution of "VACUUM FREEZE;
==1526248== VALGRINDERROR-END
{
<insert_a_suppression_name_here>
Memcheck:Addr8
fun:TidStoreGetBlockOffsets
fun:lazy_vacuum_heap_rel
fun:lazy_vacuum
fun:lazy_scan_heap
fun:heap_vacuum_rel
fun:table_relation_vacuum
fun:vacuum_rel
fun:vacuum
fun:ExecVacuum
fun:standard_ProcessUtility
fun:ProcessUtility
fun:PortalRunUtility
fun:PortalRunMulti
}
==152
[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2025-02-14%2018%3A00%3A12
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: pgsql: Use streaming read I/O in VACUUM's third phase
@ 2025-02-14 20:29 Melanie Plageman <melanieplageman@gmail.com>
parent: Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 0 replies; 4+ messages in thread
From: Melanie Plageman @ 2025-02-14 20:29 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org; +Cc: Thomas Munro <thomas.munro@gmail.com>
On Fri, Feb 14, 2025 at 1:31 PM Melanie Plageman
<melanieplageman@gmail.com> wrote:
>
> On Fri, Feb 14, 2025 at 12:59 PM Melanie Plageman
> <melanieplageman@gmail.com> wrote:
> >
> > Use streaming read I/O in VACUUM's third phase
> >
> > Make vacuum's third phase (its second pass over the heap), which reaps
> > dead items collected in the first phase and marks them as reusable, use
> > the read stream API. This commit adds a new read stream callback,
> > vacuum_reap_lp_read_stream_next(), that looks ahead in the TidStore and
> > returns the next block number to read for vacuum.
> >
> > Author: Melanie Plageman <melanieplageman@gmail.com>
> > Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
> > Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
> > Discussion: https://postgr.es/m/CA%2BhUKGKN3oy0bN_3yv8hd78a4%2BM1tJC9z7mD8%2Bf%2ByA%2BGeoFUwQ%40mail.gmail.com
> >
> > Branch
> > ------
> > master
> >
> > Details
> > -------
> > https://git.postgresql.org/pg/commitdiff/c3e775e608f2a6d0bcfba147bf08a506827cc567
> >
> > Modified Files
> > --------------
> > src/backend/access/heap/vacuumlazy.c | 55 ++++++++++++++++++++++++++++++++----
> > 1 file changed, 49 insertions(+), 6 deletions(-)
>
> I'm looking into the valgrind failures [1].
>
> ==1526248== VALGRINDERROR-END
> {
> <insert_a_suppression_name_here>
> Memcheck:Addr1
> fun:lazy_scan_heap
> fun:heap_vacuum_rel
> fun:table_relation_vacuum
> fun:vacuum_rel
> fun:vacuum_rel
> fun:vacuum
> fun:ExecVacuum
> fun:standard_ProcessUtility
> fun:ProcessUtility
> fun:PortalRunUtility
> fun:PortalRunMulti
> fun:PortalRun
> fun:exec_simple_query
> }
Looks like there is something wrong with the read stream API. This is
the first read stream user taking advantage of per_buffer_data. Thomas
and I are investigating further. It is trivially reproducible when
running intidb under valgrind.
- Melanie
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2025-02-14 20:29 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-02-14 17:59 pgsql: Use streaming read I/O in VACUUM's first phase Melanie Plageman <melanieplageman@gmail.com>
2025-02-14 17:59 pgsql: Use streaming read I/O in VACUUM's third phase Melanie Plageman <melanieplageman@gmail.com>
2025-02-14 18:31 ` Re: pgsql: Use streaming read I/O in VACUUM's third phase Melanie Plageman <melanieplageman@gmail.com>
2025-02-14 20:29 ` Re: pgsql: Use streaming read I/O in VACUUM's third phase Melanie Plageman <melanieplageman@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox