public inbox for [email protected]  

help / color / mirror / Atom feed

From: TIM CHILD <[email protected]>
To: Onni Hakala <[email protected]>
To: [email protected]
Subject: Re: Recommendations on how to combine SSD and HDD drives in bare metal PostgreSQL server
Date: Wed, 16 Oct 2024 09:01:46 -0700 (PDT)
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>

Omni,

Firstly, it looks like to have a solution, ( a fixed number and type of disks) looking for a problem. Its better to consider what is right mix of disks for your application the server. 

To understand the best physical layout, you need to know the logical  access patterns to your data. When you access your data are you doing random queries, or index scans or table scans or  computing aggregates? What queries are most important and what response time are you targeting? 
Once you have an idea of your logical access patterns you map that to you physical layout.  

Postgres Tablespaces are useful as they allow you locate tables  and indexes of different physical devices. For example it will be better to put small frequently used tables of SSD's. Or just put frequently used indexes on the SSD.  

There are many factors to consider when planning the physical storage layout. There could me a need for AI apps to do just that!

-Tim


> On 10/16/2024 7:06 AM PDT Onni Hakala <[email protected]> wrote:
> 
>  
> Hey,
> 
> I have a large dataset of > 100TB which would be very expensive to store solely into SSD drives.
> 
> I have access to a server which has 2x 3.84TB NVME SSD disks and large array of HDD drives 8 x 22TB.
> 
> Most of the data that I have in my dataset is very rarely accessed and is stored only for archival purposes.
> 
> What would be the de-facto way to use both SSD and HDD together in a way use where commonly used data would be fast to access and old data would eventually only be stored in compressed format in the HDDs?
> 
> I was initially looking into building zpool using zfs with raidz3 and zstd compression for my HDDs but I’m unsure how to add the SSDs into this equation and I thought that this is probably a common scenario and wanted to ask opinions from here.
> 
> Thanks in advance,
> Onni Hakala

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: Recommendations on how to combine SSD and HDD drives in bare metal PostgreSQL server
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox