pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Justin Pryzby <pryzby@telsasoft.com>
To: pgsql-hackers@postgresql.org
Cc: Amit Kapila <akapila@postgresql.org>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>
Subject: Re: pg15b3: crash in paralell vacuum
Date: Thu, 18 Aug 2022 09:04:15 -0500
Message-ID: <20220818140415.GN26426@telsasoft.com> (raw)
In-Reply-To: <20220818133406.GL26426@telsasoft.com>
References: <20220818133406.GL26426@telsasoft.com>

On Thu, Aug 18, 2022 at 08:34:06AM -0500, Justin Pryzby wrote:
> Unfortunately, it looks like the RPM packages are compiled with -O2, so this is
> of limited use.  So I'll be back shortly with more...

#3  0x00000000006874f1 in parallel_vacuum_process_all_indexes (pvs=0x25bdce0, num_index_scans=0, vacuum=vacuum@entry=false) at vacuumparallel.c:611
611                     Assert(indstats->status == PARALLEL_INDVAC_STATUS_INITIAL);

(gdb) p *pvs
$1 = {pcxt = 0x25bc1e0, indrels = 0x25bbf70, nindexes = 8, shared = 0x7fc5184393a0, indstats = 0x7fc5184393e0, dead_items = 0x7fc5144393a0, buffer_usage = 0x7fc514439280, wal_usage = 0x7fc514439240, 
  will_parallel_vacuum = 0x266d818, nindexes_parallel_bulkdel = 5, nindexes_parallel_cleanup = 0, nindexes_parallel_condcleanup = 5, bstrategy = 0x264f120, relnamespace = 0x0, relname = 0x0, indname = 0x0, 
  status = PARALLEL_INDVAC_STATUS_INITIAL}

(gdb) p *indstats
$2 = {status = 11, parallel_workers_can_process = false, istat_updated = false, istat = {num_pages = 0, estimated_count = false, num_index_tuples = 0, tuples_removed = 0, pages_newly_deleted = 0, pages_deleted = 1, 
    pages_free = 0}}

(gdb) bt f
...
#3  0x00000000006874f1 in parallel_vacuum_process_all_indexes (pvs=0x25bdce0, num_index_scans=0, vacuum=vacuum@entry=false) at vacuumparallel.c:611
        indstats = 0x7fc5184393e0
        i = 0
        nworkers = 2
        new_status = PARALLEL_INDVAC_STATUS_NEED_CLEANUP
        __func__ = "parallel_vacuum_process_all_indexes"
#4  0x0000000000687ef0 in parallel_vacuum_cleanup_all_indexes (pvs=<optimized out>, num_table_tuples=num_table_tuples@entry=409149, num_index_scans=<optimized out>, estimated_count=estimated_count@entry=true)
    at vacuumparallel.c:486
No locals.
#5  0x00000000004f80b8 in lazy_cleanup_all_indexes (vacrel=vacrel@entry=0x25bc510) at vacuumlazy.c:2679
        reltuples = 409149
        estimated_count = true
#6  0x00000000004f884a in lazy_scan_heap (vacrel=vacrel@entry=0x25bc510) at vacuumlazy.c:1278
        rel_pages = 67334
        blkno = 67334
        next_unskippable_block = 67334
        next_failsafe_block = 0
        next_fsm_block_to_vacuum = 0
        dead_items = 0x7fc5144393a0
        vmbuffer = 1300
        next_unskippable_allvis = true
        skipping_current_range = false
        initprog_index = {0, 1, 5}
        initprog_val = {1, 67334, 11184809}
        __func__ = "lazy_scan_heap"
#7  0x00000000004f925f in heap_vacuum_rel (rel=0x7fc52df6b820, params=0x7ffd74f74620, bstrategy=0x264f120) at vacuumlazy.c:534
        vacrel = 0x25bc510
        verbose = true
        instrument = <optimized out>
        aggressive = false
        skipwithvm = true
        frozenxid_updated = false
        minmulti_updated = false
        OldestXmin = 32759288
        FreezeLimit = 4277726584
        OldestMxact = 157411
        MultiXactCutoff = 4290124707
        orig_rel_pages = 67334
        new_rel_pages = <optimized out>
        new_rel_allvisible = 4
        ru0 = {tv = {tv_sec = 1660830451, tv_usec = 473980}, ru = {ru_utime = {tv_sec = 0, tv_usec = 317891}, ru_stime = {tv_sec = 1, tv_usec = 212372}, {ru_maxrss = 74524, __ru_maxrss_word = 74524}, {ru_ixrss = 0, 
              __ru_ixrss_word = 0}, {ru_idrss = 0, __ru_idrss_word = 0}, {ru_isrss = 0, __ru_isrss_word = 0}, {ru_minflt = 18870, __ru_minflt_word = 18870}, {ru_majflt = 0, __ru_majflt_word = 0}, {ru_nswap = 0, 
              __ru_nswap_word = 0}, {ru_inblock = 1124750, __ru_inblock_word = 1124750}, {ru_oublock = 0, __ru_oublock_word = 0}, {ru_msgsnd = 0, __ru_msgsnd_word = 0}, {ru_msgrcv = 0, __ru_msgrcv_word = 0}, {ru_nsignals = 0, 
              __ru_nsignals_word = 0}, {ru_nvcsw = 42, __ru_nvcsw_word = 42}, {ru_nivcsw = 35, __ru_nivcsw_word = 35}}}
        starttime = 714145651473980
        startreadtime = 0
        startwritetime = 0
        startwalusage = {wal_records = 2, wal_fpi = 0, wal_bytes = 421}
        StartPageHit = 50
        StartPageMiss = 0
        StartPageDirty = 0
        errcallback = {previous = 0x0, callback = 0x4f5f41 <vacuum_error_callback>, arg = 0x25bc510}
        indnames = 0x266d838
        __func__ = "heap_vacuum_rel"

This is a qemu VM which (full disclosure) has crashed a few times recently due
to OOM.  This is probably a postgres bug, but conceivably it's being tickled by
bad data (although the vm crashing shouldn't cause that, either, following
recovery).  This is also an instance that was pg_upgraded from v14 (and earlier
versions) to v15b1 and then b2, so it's conceivably possible there's weird data
pages that wouldn't be written by beta3.  But that doesn't seem to be the issue
here anyway.

-- 
Justin





view thread (7+ messages)  latest in thread

Message-ID: <20220818140415.GN26426@telsasoft.com>
Permalink:  ../20220818140415.GN26426@telsasoft.com/
Also on:    postgresql.org/message-id/20220818140415.GN26426@telsasoft.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-hackers@postgresql.org
  Cc: pryzby@telsasoft.com, akapila@postgresql.org, sawada.mshk@gmail.com
  Subject: Re: pg15b3: crash in paralell vacuum
  In-Reply-To: <20220818140415.GN26426@telsasoft.com>
  References: <20220818133406.GL26426@telsasoft.com> <20220818140415.GN26426@telsasoft.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox