public inbox for [email protected]
help / color / mirror / Atom feedFrom: Steve Midgley <[email protected]>
To: Shaozhong SHI <[email protected]>
Cc: pgsql-sql <[email protected]>
Subject: Re: parallel-processing multiple similar query tasks - any example?
Date: Thu, 28 Apr 2022 10:14:47 -0700
Message-ID: <CAJexoSJy1yXa9vQuoN0U8ykf0w29yWJWw9aODak6sHsx+wzjNw@mail.gmail.com> (raw)
In-Reply-To: <CA+i5JwZ1_gu0g-X+5P8hVhyc1YS7F4nhrcEQdZKADOp+7o+1SQ@mail.gmail.com>
References: <CA+i5JwaC9Z-rjWiyo7LpncdK=CuQcKzuVNMnAhPW1oi=8c9Sug@mail.gmail.com>
<CA+i5JwZ1_gu0g-X+5P8hVhyc1YS7F4nhrcEQdZKADOp+7o+1SQ@mail.gmail.com>
On Wed, Apr 27, 2022 at 4:34 PM Shaozhong SHI <[email protected]>
wrote:
>
>
>
> multiple similar query tasks are as follows:
>
> select * from a_table where country ='UK'
> select * from a_table where country='France'
> and so on
>
> How best to parallel-processing such types of multiple similar query tasks?
>
>
> This depends on how you are engaging with the queries when they return
results. Let's assume you are running them from a programming environment
with an ORM layer. In that case you can run each query in a separate thread
and connection pipe, and the queries will run async just fine. If you are
running at the command line using psql, you can just open multiple shells
and run each query from a different shell terminal.
Postgres is very good at async queries, so your challenge is really
figuring out how you will use the results of each query and setting up the
environment sending the queries to perform asynchronously.
Steve
view thread (4+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: parallel-processing multiple similar query tasks - any example?
In-Reply-To: <CAJexoSJy1yXa9vQuoN0U8ykf0w29yWJWw9aODak6sHsx+wzjNw@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox