Subject: Postgresql Database and PG_WAL locations

From: Henry Ashu <[email-masked]>
Date: Thu, 29 Aug 2024 11:18:02 -0700

I have a database that's about 2TB in size, and I want to place the
database files in a separate mount point(PGDATA) and the log files in a
different mount point(PG_WAL). What's your take on this?.

Thank you

*Henry Ashu*

Database Administrator
*o:* 503.672.5114 | *f:* 800.551.8821 | DAT.com
 <https://www.dat.com/>    [image: Like us on
Facebook] <https://www.facebook.com/loadboards>  [image: Follow us on
Twitter] <https://twitter.com/loadboards>  [image: Connect with us on
LinkedIn] <https://www.linkedin.com/company/datsolutions>  [image: Visit
our YouTube Channel] <https://www.youtube.com/datloadboards>  [image: Visit
us on Instagram] <https://instagram.com/dat_solutions>

---

From: Matthew Tice <[email-masked]>
Date: Thu, 29 Aug 2024 12:21:38 -0600

> On Aug 29, 2024, at 12:18 PM, Henry Ashu <[email-masked]> wrote:
> 
> I have a database that's about 2TB in size, and I want to place the database files in a separate mount point(PGDATA) and the log files in a different mount point(PG_WAL). What's your take on this?.

Take a look at https://wiki.postgresql.org/wiki/Installation_and_Administration_Best_practices

Essentially, yes, you will want your WAL and data stored on different devices (or the very least, different partitions).

> Thank you
> 
> Henry Ashu
> 
> Database Administrator
> o: 503.672.5114 | f: 800.551.8821 | DAT.com
> 
>  <https://www.dat.com/>     <https://www.facebook.com/loadboards>   <https://twitter.com/loadboards>   <https://www.linkedin.com/company/datsolutions>   <https://www.youtube.com/datloadboards>   <https://instagram.com/dat_solutions>

---

From: Ron Johnson <[email-masked]>
Date: Thu, 29 Aug 2024 15:02:54 -0400

On Thu, Aug 29, 2024 at 2:22 PM Matthew Tice <[email-masked]> wrote:

> Take a look at
> https://wiki.postgresql.org/wiki/Installation_and_Administration_Best_practices
>
> Essentially, yes, you will want your WAL and data stored on different
> devices (or the very least, different partitions).
>

Is that recommendation still valid?  After all, that was written when *15
years old* Sun Studio 12 was still pertinent.  Times have changed since
then.  Disks are much, *much* bigger.

-- 
Death to America, and butter sauce.
Iraq lobster!

---

From: Matthew Tice <[email-masked]>
Date: Thu, 29 Aug 2024 13:11:57 -0600

> On Aug 29, 2024, at 1:02 PM, Ron Johnson <[email-masked]> wrote:
> 
> On Thu, Aug 29, 2024 at 2:22 PM Matthew Tice <[email-masked] <mailto:[email-masked]>> wrote:
>>> On Aug 29, 2024, at 12:18 PM, Henry Ashu <[email-masked] <mailto:[email-masked]>> wrote:
>>> 
> Is that recommendation still valid?  After all, that was written when 15 years old Sun Studio 12 was still pertinent.  Times have changed since then.  Disks are much, much bigger.

Good point.  I didn’t even notice that disclaimer at the top "Seek knowledge elsewhere, as the material in here predates Postgres 9 it seems.”…

But regarding the wal/data separation, I still think it’s relevant (isolating the IO between data files and wal).  But I guess like anything else it depends on your workload (and, in this case, your storage subsystem capabilities).

---

From: Scott Ribe <[email-masked]>
Date: Thu, 29 Aug 2024 13:14:42 -0600

> Is that recommendation still valid?  After all, that was written when 15 years old Sun Studio 12 was still pertinent.  Times have changed since then.  Disks are much, much bigger.

We still have WAL and data having the same data written (at somewhat different times), so it's still useful to give them separate bandwidth to storage--for databases with high write loads.

---

From: Laurenz Albe <[email-masked]>
Date: Mon, 02 Sep 2024 12:30:32 +0200

On Thu, 2024-08-29 at 15:02 -0400, Ron Johnson wrote:
> > > I have a database that's about 2TB in size, and I want to place the database
> > > files in a separate mount point(PGDATA) and the log files in a different mount
> > > point(PG_WAL). What's your take on this?.
> > 
> > 
> > Essentially, yes, you will want your WAL and data stored on different devices
> > (or the very least, different partitions).
> 
> Is that recommendation still valid?  After all, that was written when 15 years old
> Sun Studio 12 was still pertinent.  Times have changed since then.  Disks are much, much bigger.

I think the advice is still valid.

Today you'd have different filesystems on different logical volumes rather
than different physical disks, but it is still a good idea to separate data and WAL,
so that they cannot fill up each other's file system.

I'd actually define a third file system for the PostgreSQL log files, for
the same reason.

Yours,
Laurenz Albe

---

From: Ron Johnson <[email-masked]>
Date: Mon, 2 Sep 2024 09:15:57 -0400

On Mon, Sep 2, 2024 at 6:30 AM Laurenz Albe <[email-masked]>
wrote:

> database
> devices
> than different physical disks,

None of our disks are physical; they're all SAN LUNs.

> but it is still a good idea to separate data and WAL,

Regular checkpoints, transactions(*) that don't stay open for hours or
days, and monitoring replication to ensure that it keeps replicating data
instead of piling up on the primary server all solve that problem

Honestly... it's been *YEARS* since I've seen that problem.

Besides, "disks are cheap", right?

*COPY statements don't count.

> the same reason.

I set log_directory to /var/log/postgresql because logs go in
/var/log.  😀  (pg_basebackup won't replicate it, which is also handy.) On
a separate partition so as to isolate PG data from other application and OS
data (and lets me manage capacity via a script).

Ditto the PgBackrest directory: isolate PG data from other application and
OS data.

We'd have to put them on separate mount points anyway, since the VM build
process doesn't like large / and /boot disks.

---

From: Laurenz Albe <[email-masked]>
Date: Mon, 02 Sep 2024 15:38:17 +0200

On Mon, 2024-09-02 at 09:15 -0400, Ron Johnson wrote:
> Regular checkpoints, transactions(*) that don't stay open for hours
> or days, and monitoring replication to ensure that it keeps replicating
> data instead of piling up on the primary server all solve that problem

Right, if everything is working as it should, monitoring is in place
etc. this will never happen.

But I believe that paranoia is a virtue for the DBA, and it is better
to have a second line of defense than a crash of the database.

> Honestly... it's been YEARS since I've seen that problem.

I see it all the time (on systems I didn't set up).

Yours,
Laurenz Albe