Feedback-ID: i76984098:Fastmail
Message-ID: <6d7e1022-6404-4dab-8467-8d1f6e8b63cb@aklaver.com>
Date: Wed, 5 Mar 2025 08:08:51 -0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Quesion about querying distributed databases
To: me nefcanto <sn.1361@gmail.com>, Laurenz Albe <laurenz.albe@cybertec.at>
Cc: pgsql-general@lists.postgresql.org
References: <CAEHBEOBuoMFWuhHM3L_Zr6o1enELju-Vns6Pknt4TT+6MFQOwQ@mail.gmail.com>
 <fd47b28c-8f6b-4bfb-a393-160c5c3de8c0@aklaver.com>
 <CAEHBEOD969YrbPH_z9OEmThWx3-w4sMMaHLhZLOQwqCwE8Y58Q@mail.gmail.com>
 <caf4e99941b3c83bb9eab91e33b144b826b68f79.camel@cybertec.at>
 <CAEHBEOBXzkGTqxQSYqmEFN5hbc=zsGWFpU9h8zf7AAPv4VdOWQ@mail.gmail.com>
 <c9699a6fd331d33864fc269060d8d961f784d827.camel@cybertec.at>
 <CAEHBEOBNoG8RkKuCcQQWkbYppMLMzA0MXq+s0kZ6wKWgD7+45Q@mail.gmail.com>
 <099b49ebae94e23f19afdad3f8c9c6e702a3a2d5.camel@cybertec.at>
 <CAEHBEODw8svX557pjB_EL-Os7KWtwi-9Uq=RuCkRKgHVZWw8Bw@mail.gmail.com>
Content-Language: en-US
From: Adrian Klaver <adrian.klaver@aklaver.com>
In-Reply-To: <CAEHBEODw8svX557pjB_EL-Os7KWtwi-9Uq=RuCkRKgHVZWw8Bw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/6d7e1022-6404-4dab-8467-8d1f6e8b63cb%40aklaver.com>
Precedence: bulk

On 3/5/25 04:15, me nefcanto wrote:
> Dear Laurenz, the point is that I think if we put all databases into one 
> database, then we have blocked our growth in the future.

How?

> A monolith database can be scaled only vertically. We have had huge 
> headaches in the past with SQL Server on Windows and a single database.
> But when you divide bounded contexts into different databases, then you 
> have the chance to deploy each database on a separate physical machine. 
> That means a lot in terms of performance. Please correct me if I am wrong.

And you add the complexity of talking across machines, as well as 
maintaining separate machines.

> 
> Let's put this physical restriction on ourselves that we have different 
> databases. What options do we have? One option that comes to my mind, is 
> to store the ID of the categories in the Products table. This means that 
> I don't need FDW anymore. And databases can be on separate machines. I 
> first query the categories database first, get the category IDs, and 
> then add a where clause to limit the product search. That could be an 
> option. Array data type in Postgres is something that I think other 
> RDBMSs do not have. Will that work? And how about attributes? Because 
> attributes are more than a single ID. I should store the attribute key, 
> alongside its value. It's a key-value pair. What can I do for that?

You seem to be going out of the way to make your life more complicated.

The only way you are going to find an answer is set up test cases and 
experiment. My bet is a single server with a single database and 
multiple schemas is where you end up, after all that is where you are 
starting from.


> 
> Thank you for sharing your time. I really appreciate it.
> Saeed
> 
> 
> 
> 
> 
> On Wed, Mar 5, 2025 at 3:18 PM Laurenz Albe <laurenz.albe@cybertec.at 
> <mailto:laurenz.albe@cybertec.at>> wrote:
> 
>     On Wed, 2025-03-05 at 14:18 +0330, me nefcanto wrote:
>      > That means a solid monolith database. We lose many goodies with that.
>      > As a real-world example, right now we can import a single database
>      > from the production to the development to test and troubleshoot data.
> 
>     Well, can't you import a single schema then?
> 
>      > What if we host all databases on the same server and use FDW. What
>      > happens in that case? Does it return 100 thousand records and join
>      > in the memory?
> 
>     It will do just the same thing.  The performance could be better
>     because of the reduced latency.
> 
>      > Because in SQL Server, when you perform a cross-database query
>      > (not cross-server) the performance is extremely good, proving that
>      > it does not return 100 thousand ItemId from Taxonomy.ItemCategories
>      > to join with ProductId.
>      >
>      > Is that the same case with Postgres too, If databases are located
>      > on one server?
> 
>     No, you cannot perform cross-database queries without a foreign
>     data wrapper.  I don't see a reason why the statement shouldn't
>     perform as well as in SQL Server if you use schemas instead of
>     databases.
> 
>     Yours,
>     Laurenz Albe
> 

-- 
Adrian Klaver
adrian.klaver@aklaver.com