Date: Thu, 6 Jan 2022 22:43:37 -0600
From: Justin Pryzby <pryzby@telsasoft.com>
To: Takashi Menjo <takashi.menjo@gmail.com>
Cc: Thomas Munro <thomas.munro@gmail.com>, pgsql-hackers@postgresql.org,
	Tomas Vondra <tomas.vondra@enterprisedb.com>,
	Masahiko Sawada <sawada.mshk@gmail.com>,
	Heikki Linnakangas <hlinnaka@iki.fi>,
	"Tsunakawa, Takayuki" <tsunakawa.takay@fujitsu.com>,
	"Deng, Gang" <gang.deng@intel.com>,
	Takashi Menjo <takashi.menjou.vg@hco.ntt.co.jp>,
	Matthias van de Meent <boekewurm+postgres@gmail.com>,
	Daniel Gustafsson <daniel@yesql.se>
Subject: Re: Map WAL segment files on PMEM as WAL buffers
Message-ID: <20220107044337.GB14051@telsasoft.com>
References: 
 <CAOwnP3N9p5byYUag4h+_7Z9+vmDgMNVs+Wuu9n2c6JUFk1jibg@mail.gmail.com>
 <CAOwnP3OaRYEgj0xCpXEJ2Pa7-52euL+gOTM8eMa+fOwpUPU4ZA@mail.gmail.com>
 <CEEFC453-353D-40B8-859F-6DF9ADF6F4BE@yesql.se>
 <CAOwnP3O6SWP6GO-o2RfkrDcF+5g3Y+K+L9Hp_6m3KSKebdm6Hg@mail.gmail.com>
 <CAOwnP3PS4fRNCOkGvSf1USv_bHSgQzMzn6CrbpbV1HM8nse1eQ@mail.gmail.com>
 <CAOwnP3N6Zi2nQ0WzuXA69w2mhZ5R5uvVJUDNH92CTA09MErh9g@mail.gmail.com>
 <20220106040000.GU14051@telsasoft.com>
 <CA+hUKGL4CjH5+ZJGwgZapj0ZSPbptCz=kHPDEu-N+JJMRANZtA@mail.gmail.com>
 <20220106051908.GV14051@telsasoft.com>
 <CAOwnP3PSDxKhBUGS=97_b44s65DxaGexJB2-Nbmf7SGiLUJ2pg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CAOwnP3PSDxKhBUGS=97_b44s65DxaGexJB2-Nbmf7SGiLUJ2pg@mail.gmail.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Archived-At: 
 <https://www.postgresql.org/message-id/20220107044337.GB14051%40telsasoft.com>
Precedence: bulk

On Fri, Jan 07, 2022 at 12:50:01PM +0900, Takashi Menjo wrote:
> > But in this case it really doesn't work :(
> >
> > running bootstrap script ... 2022-01-05 23:17:30.244 CST [12088] FATAL:  file not on PMEM: path "pg_wal/000000010000000000000001"
> 
> Do you have a real PMEM device such as NVDIMM-N or Intel Optane PMem?

No - the point is that we'd like to have a way to exercise this patch on the
cfbot.  Particularly the new code introduced by this patch, not just the
--without-pmem case...

I was able to make this pass "make check" by adding this to main() in
src/backend/main/main.c:
| setenv("PMEM_IS_PMEM_FORCE", "1", 0);

I think you should add a patch which does what Thomas suggested: 1) add to
./.cirrus.yaml installation of the libpmem package for debian/bsd/mac/windows;
2) add setenv to main(), as above; 3) change configure.ac and guc.c to default
to --with-libpmem and wal_pmem_map=on.  This should be the last patch, for
cfbot only, not meant to be merged.

You can test that the package installation part works before mailing patches to
the list with the instructions here:

src/tools/ci/README:
Enabling cirrus-ci in a github repository..

> If you don't, you have two alternatives below. Note that neither of
> them ensures durability. Each of them is just for testing.
> 2. Set the environment variable PMEM_IS_PMEM_FORCE=1 to tell libpmem
> to treat any devices as if they were PMEM.

The next revision should surely squish all the fixes into their corresponding
patches to be fixed.  Each of the patches ought to be compile and pass tests
without depending on the "following" patches:  0001 without 0002-, 0001-0002
without 0003-, etc.

-- 
Justin