Date: Sat, 5 Oct 2024 10:08:00 -0700
From: Noah Misch <noah@leadboat.com>
To: Alexander Lakhin <exclusion@gmail.com>
Cc: buildfarm-members@postgresql.org
Subject: Re: shmat() fails with Not enough space on wrasse
Message-ID: <20241005170800.29@rfd.leadboat.com>
References: <bd2934ff-ede0-65e6-b5a3-f23528194276@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <bd2934ff-ede0-65e6-b5a3-f23528194276@gmail.com>
User-Agent: Mutt/2.2.12 (2023-09-09)
Archived-At: 
 <https://www.postgresql.org/message-id/20241005170800.29%40rfd.leadboat.com>
Precedence: bulk

On Thu, Oct 03, 2024 at 07:00:00AM +0300, Alexander Lakhin wrote:
> Could you please look at sporadic test failures occurring on wrasse?
> 
> Yesterday wrasse failed pg_rewindCheck ([3]) and it's the third failure of
> the same kind since July. The failure log regress_log_001_basic contains:
> # Running: pg_rewind --debug --source-server port=17304
> host=/home/nm/farm/tmp/sbGTlHPVwD dbname='postgres' user=rewind_user --target-pgdata=/export/home/nm/farm/studio64v12_6/REL_14_STABLE/pgsql.build/src/bin/pg_rewind/tmp_check/t_001_basic_primary_remote_data/pgdata
> --no-sync --write-recovery-conf
> pg_rewind: executing ".../inst/bin/postgres" for target server to complete crash recovery
> 2024-10-03 01:32:24.149 CEST [29411:1] FATAL:  shmat(id=1392509011, addr=0, flags=0x4000) failed: Not enough space
> pg_rewind: error: postgres single-user mode in target cluster failed
> pg_rewind: fatal: Command was: ".../inst/bin/postgres" --single -F -D
> "...src/bin/pg_rewind/tmp_check/t_001_basic_primary_remote_data/pgdata"
> template1 < "/dev/null"
> not ok 11 - pg_rewind remote
> 
> I don't know SPARC Solaris, but I've found a recommendation to increase
> zone.max-locked-memory [4] in a similar situation.

I'm seeing no limit on that one and generous limits on other memory-related
parameters.  (I checked it from inside a cron job, just in case that
environment had been different.)

prctl $$ | ggrep -A4 -E '(memory|shm|^process:)'

process: 21611: sh -c { date; prctl $$ | ggrep -A4 -E '(memory|shm|^process:)'; } >/ex
NAME    PRIVILEGE       VALUE    FLAG   ACTION                       RECIPIENT
process.max-deferred-posts
        basic              32       -   deny                             21611
        privileged        100       -   deny                                 -
--
project.max-adi-metadata-memory
        usage               0B   
        system          16.0EB    max   deny                                 -
project.max-locked-memory
        usage           1.02MB   
        system          16.0EB    max   deny                                 -
project.max-port-ids
        privileged      8.19K       -   deny                                 -
--
project.max-shm-memory
        usage           1.02MB   
        privileged      7.85GB      -   deny                                 -
        system          16.0EB    max   deny                                 -
project.max-shm-ids
        privileged        128       -   deny                                 -
        system          16.8M     max   deny                                 -
project.max-msg-ids
        privileged        128       -   deny                                 -
--
project.max-crypto-memory
        usage               0B   
        privileged      7.85GB      -   deny                                 -
        system          16.0EB    max   deny                                 -
project.max-tasks
--
zone.max-adi-metadata-memory
        usage               0B   
        system          16.0EB    max   deny                                 -
zone.max-locked-memory
        usage           1.02MB   
        system          16.0EB    max   deny                                 -
zone.max-mrp-ids
        usage               0    
--
zone.max-shm-memory
        usage           1.02MB   
        system          16.0EB    max   deny                                 -
zone.max-shm-ids
        usage               3    
        system          16.8M     max   deny                                 -
zone.max-sem-ids
        usage               8